{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Tce3stUlHN0L"
      },
      "source": [
        "##### Copyright 2024 Google LLC."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "cellView": "form",
        "id": "tuOe1ymfHZPu"
      },
      "outputs": [],
      "source": [
        "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "# https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yeadDkMiISin"
      },
      "source": [
        "# PaLM API: Tuning Quickstart with Python"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lEXQ3OwKIa-O"
      },
      "source": [
        "<table class=\"tfo-notebook-buttons\" align=\"left\">\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://ai.google.dev/palm_docs/tuning_quickstart_python\"><img src=\"https://ai.google.dev/static/site-assets/images/docs/notebook-site-button.png\" height=\"32\" width=\"32\" />View on ai.google.dev</a>\n",
        "  </td>\n",
        "    <td>\n",
        "    <a target=\"_blank\" href=\"https://colab.research.google.com/github/google/generative-ai-docs/blob/main/site/en/palm_docs/tuning_quickstart_python.ipynb\"><img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" />Run in Google Colab</a>\n",
        "  </td>\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://github.com/google/generative-ai-docs/blob/main/site/en/palm_docs/tuning_quickstart_python.ipynb\"><img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" />View source on GitHub</a>\n",
        "  </td>\n",
        "    <td>\n",
        "    <a target=\"_blank\" href=\"https://ai.google.dev/palm_docs/tuning_quickstart_python\"><img src=\"https://www.tensorflow.org/images/download_logo_32px.png\" />Download notebook</a>\n",
        "  </td>\n",
        "</table>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Jp_CKyzxUqx6"
      },
      "source": [
        "In this notebook, you'll learn how to get started with the tuning service using the Python client library for the PaLM API. Here, you'll learn how to tune the text model behind the PaLM API's text generation service."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4JXd-HdCsKdZ"
      },
      "source": [
        "**Note**: At this time, tuning is only available for the `text-bison-001` model."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sOz_wyZAlCuQ"
      },
      "source": [
        "## Setup"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SWxKvwd-MSIV"
      },
      "source": [
        "### Authenticate"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JjS8Zy1ojIgc"
      },
      "source": [
        "The PaLM API lets you tune models on your own data. Since it's your data and\n",
        "your tuned models this needs stricter access controls than API-Keys can provide.\n",
        "\n",
        "Before you can run this tutorial, you'll need to\n",
        "[setup OAuth for your project](oauth_quickstart.ipynb).\n",
        "\n",
        "If you want to run this notebook in Colab start by uploading your\n",
        "`client_secret*.json` file using the \"File > Upload\" option.\n",
        "\n",
        "<img width=400 src=\"https://ai.google.dev/tutorials/images/colab_upload.png\">"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "JjS834433jIgc"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "client_secret.json"
          ]
        }
      ],
      "source": [
        "!cp client_secret*.json client_secret.json\n",
        "!ls client_secret.json"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "I6zTC-3mJ0-2"
      },
      "source": [
        "This gcloud command turns the `client_secret.json` file into credentials that can be used to authenticate with the service.\n",
        "\n",
        "Important: If you're running this in Colab, **don't just click the link it prints**. That will fail. Follow the instriuctions and copy the `gcloud` command it prints to your local machine and run it there, then paste the output from your local machine back here."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "9FUwyB_MJ0-2"
      },
      "outputs": [],
      "source": [
        "import os\n",
        "if 'COLAB_RELEASE_TAG' in os.environ:\n",
        "  # Use `--no-browser` in colab\n",
        "  !gcloud auth application-default login --no-browser --client-id-file client_secret.json --scopes='https://www.googleapis.com/auth/cloud-platform,https://www.googleapis.com/auth/generative-language.tuning'\n",
        "else:\n",
        "  !gcloud auth application-default login --client-id-file client_secret.json --scopes='https://www.googleapis.com/auth/cloud-platform,https://www.googleapis.com/auth/generative-language.tuning'"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aHimx8NGMWDj"
      },
      "source": [
        "### Install the client library"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "cbcf72bcb56d"
      },
      "outputs": [],
      "source": [
        "!pip install -q google-generativeai"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jdIYSl2kN0cq"
      },
      "source": [
        "### Import libraries"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "8enrppafJPCX"
      },
      "outputs": [],
      "source": [
        "import google.generativeai as genai\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "P-MYZECwlRCq"
      },
      "source": [
        "You can check you existing tuned models with the `genai.list_tuned_model` method."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "XyWzoYFxU4r6"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "tunedModels/my-model-8527\n",
            "tunedModels/my-model-7092\n",
            "tunedModels/my-model-2778\n",
            "tunedModels/my-model-1298\n",
            "tunedModels/my-model-3883\n"
          ]
        }
      ],
      "source": [
        "for i, m in zip(range(5), genai.list_tuned_models()):\n",
        "  print(m.name)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BhkXRzciv3Dp"
      },
      "source": [
        "## Create tuned model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OO8VZYAinLWc"
      },
      "source": [
        "To create a tuned model, you need to pass your dataset to the model in the `genai.create_tuned_model` method. You can do this be directly defining the input and output values in the call or importing from a file into a dataframe to pass to the method.\n",
        "\n",
        "For this example, you will tune a model to generate the next number in the sequence. For example, if the input is `1`, the model should output `2`. If the input is `one hundred`, the output should be `one hundred one`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "w-EBSe9wTbLB"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'models/text-bison-001'"
            ]
          },
          "execution_count": 11,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "base_model = [\n",
        "    m for m in genai.list_models()\n",
        "    if \"createTunedTextModel\" in m.supported_generation_methods][0]\n",
        "base_model.name"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "baHjHh1oTTTC"
      },
      "outputs": [],
      "source": [
        "import random\n",
        "\n",
        "name = f'generate-num-{random.randint(0,10000)}'\n",
        "operation = genai.create_tuned_model(\n",
        "    # You can use a tuned model here too. Set `source_model=\"tunedModels/...\"`\n",
        "    source_model=base_model.name,\n",
        "    training_data=[\n",
        "        {\n",
        "             'text_input': '1',\n",
        "             'output': '2',\n",
        "        },{\n",
        "             'text_input': '3',\n",
        "             'output': '4',\n",
        "        },{\n",
        "             'text_input': '-3',\n",
        "             'output': '-2',\n",
        "        },{\n",
        "             'text_input': 'twenty two',\n",
        "             'output': 'twenty three',\n",
        "        },{\n",
        "             'text_input': 'two hundred',\n",
        "             'output': 'two hundred one',\n",
        "        },{\n",
        "             'text_input': 'ninety nine',\n",
        "             'output': 'one hundred',\n",
        "        },{\n",
        "             'text_input': '8',\n",
        "             'output': '9',\n",
        "        },{\n",
        "             'text_input': '-98',\n",
        "             'output': '-97',\n",
        "        },{\n",
        "             'text_input': '1,000',\n",
        "             'output': '1,001',\n",
        "        },{\n",
        "             'text_input': '10,100,000',\n",
        "             'output': '10,100,001',\n",
        "        },{\n",
        "             'text_input': 'thirteen',\n",
        "             'output': 'fourteen',\n",
        "        },{\n",
        "             'text_input': 'eighty',\n",
        "             'output': 'eighty one',\n",
        "        },{\n",
        "             'text_input': 'one',\n",
        "             'output': 'two',\n",
        "        },{\n",
        "             'text_input': 'three',\n",
        "             'output': 'four',\n",
        "        },{\n",
        "             'text_input': 'seven',\n",
        "             'output': 'eight',\n",
        "        }\n",
        "    ],\n",
        "    id = name,\n",
        "    epoch_count = 100,\n",
        "    batch_size=4,\n",
        "    learning_rate=0.001,\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-As7ayWDK1w8"
      },
      "source": [
        "Your tuned model is immediately added to the list of tuned models, but its status is set to \"creating\" while the model is tuned."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "su64KgY4Uztj"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "TunedModel(name='tunedModels/generate-num-9028',\n",
            "           source_model='tunedModels/generate-num-4110',\n",
            "           base_model='models/text-bison-001',\n",
            "           display_name='',\n",
            "           description='',\n",
            "           temperature=0.7,\n",
            "           top_p=0.95,\n",
            "           top_k=40,\n",
            "           state=<State.CREATING: 1>,\n",
            "           create_time=datetime.datetime(2023, 9, 29, 21, 37, 32, 188028, tzinfo=datetime.timezone.utc),\n",
            "           update_time=datetime.datetime(2023, 9, 29, 21, 37, 32, 188028, tzinfo=datetime.timezone.utc),\n",
            "           tuning_task=TuningTask(start_time=datetime.datetime(2023, 9, 29, 21, 37, 32, 734118, tzinfo=datetime.timezone.utc),\n",
            "                                  complete_time=None,\n",
            "                                  snapshots=[],\n",
            "                                  hyperparameters=Hyperparameters(epoch_count=100,\n",
            "                                                                  batch_size=4,\n",
            "                                                                  learning_rate=0.001)))\n"
          ]
        }
      ],
      "source": [
        "model = genai.get_tuned_model(f'tunedModels/{name}')\n",
        "\n",
        "model"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "EUodUwZkKPi-"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "<State.CREATING: 1>"
            ]
          },
          "execution_count": 14,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "model.state"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Pi8X5vkQv-3_"
      },
      "source": [
        "### Check tuning progress"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tWI-vAh4LJIz"
      },
      "source": [
        "Use `metadata` to check the state:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "g08vqtxYLMxT"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "tuned_model: \"tunedModels/generate-num-9028\"\n",
              "total_steps: 375"
            ]
          },
          "execution_count": 41,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "operation.metadata"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3lQ6gSMgK-kz"
      },
      "source": [
        "Wait for the training to finish using `operation.result()`, or `operation.wait_bar()`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "SOUowIv1HgSE"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "98e4b6958bfc43c98e6e77354f7bf315",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/375 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "import time\n",
        "\n",
        "for status in operation.wait_bar():\n",
        "  time.sleep(30)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4cg868HzqOx5"
      },
      "source": [
        "You can cancel your tuning job any time using the `cancel()` method. Uncomment the line below and run the code cell to cancel your job before it finishes."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "oQuJ70_hqJi9"
      },
      "outputs": [],
      "source": [
        "# operation.cancel()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lqiL0TWDqAPn"
      },
      "source": [
        "Once the tuning is complete, you can view the loss curve from the tuning results. The [loss curve](https://ai.google.dev/gemini-api/docs/model-tuning#recommended_configurations) shows how much the model's predictions deviate from the ideal outputs."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "bIiG57xWLhP7"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "<Axes: xlabel='epoch', ylabel='mean_loss'>"
            ]
          },
          "execution_count": 17,
          "metadata": {},
          "output_type": "execute_result"
        },
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAioAAAGwCAYAAACHJU4LAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABRTElEQVR4nO3de3hcVb0+8HetvffsmdzbXFuaXmjBQm8WCrVcRAVURED0gGI9VuA55yDFcvGoVA78FA6mh/Poo4inXh4P6KPc9AjeEIRCQZBCWym2tJSW1jbQJmmuM5nM7Jm99/r9MZdkkkkymU4yM8n7eZ6xzcyeyZqVyrxZl+8SSikFIiIiogIk890AIiIiouEwqBAREVHBYlAhIiKigsWgQkRERAWLQYWIiIgKFoMKERERFSwGFSIiIipYer4bcDxc18WRI0dQXl4OIUS+m0NEREQZUEohEAhg5syZkHLkMZOiDipHjhxBY2NjvptBREREWWhubsasWbNGvKaog0p5eTmA2ButqKjIc2uIiIgoE36/H42NjcnP8ZEUdVBJTPdUVFQwqBARERWZTJZtcDEtERERFSwGFSIiIipYDCpERERUsBhUiIiIqGAxqBAREVHBYlAhIiKigsWgQkRERAWLQYWIiIgKFoMKERERFSwGFSIiIipYDCpERERUsPIaVBzHwe2334558+bB5/Nh/vz5uOuuu6CUymeziIiIqEDk9VDC//qv/8LGjRvxs5/9DIsWLcK2bdtw9dVXo7KyEuvWrctn04iIiKgA5DWo/PWvf8Vll12Giy++GAAwd+5cPPTQQ3j11VfTXm9ZFizLSn7t9/snpJ0jCUcdeA0t380gIiKalPI69XPWWWdh06ZNeOuttwAAr7/+Ol588UVcdNFFaa9vampCZWVl8tbY2DiRzR3iWMDCnqN+2I6b13YQERFNVnkdUbn11lvh9/uxcOFCaJoGx3Fw9913Y/Xq1WmvX79+PW655Zbk136/P29hxXEV3unqQ08oCst2oWtcl0xERJRreQ0qjz76KH75y1/iwQcfxKJFi7Bjxw7cdNNNmDlzJtasWTPketM0YZpmHlo6VEfQwrFAbBrKsl2UFkaziIiIJpW8BpWvfOUruPXWW/GZz3wGALBkyRIcOnQITU1NaYNKoXBchebOPuhSIuq6iNic+iEiIhoPeZ2v6Ovrg5SpTdA0Da5b2B/8Hb0WOnojmFZiAADCUTvPLSIiIpqc8jqicskll+Duu+/G7NmzsWjRIrz22mv4zne+g2uuuSafzRqR4yo0d/XB0CR0TcKQEr0WgwoREdF4yGtQ+f73v4/bb78d119/Pdra2jBz5kz827/9G+644458NmtE7fHRlNqy2KIUjy7RazlQSkEIkefWERERTS5CFXEZWL/fj8rKSvT09KCiomLcv5/tuNjR3I1A2EZNPKj0RWxYtosz501nPRUiIqIMjOXzm3tqx6C9N4KOXgvTSjzJ+zyaRMRxYXFBLRERUc4xqIxBqz8MQ9Ogyf4pHl2TcFwFy3by2DIiIqLJiUElQxHbRSAchc+TZnpHAVaUIypERES5xqCSoVDUQdh24dWHdpkmBfoi3PlDRESUawwqGQpFHDhO+lL5Hk0iEGZQISIiyjUGlQz1WlFImX77saFLhG0XUR5OSERElFMMKhnq7ovCo6XffuzRJKK2w50/REREOcagkoFw1EEo6sBrpO8uQxOIOgpWlDt/iIiIcolBJQPhqAMr6sLU04+oCCGgAI6oEBER5RiDSgb6Ig4cpVLqpwwmRSzQEBERUe4wqGQgaNmQo5zjY0ju/CEiIso1BpVRKKXQHYrCTFM/ZSBdk+izHLhu0R6dREREVHAYVEZh2S76Ija8w6xPSYid+eMgwi3KREREOcOgMoq+SHwh7TA7fhI8uozv/GFQISIiyhUGlVH0RWwoqFHXqGhSwFE8nJCIiCiXGFRGEQjb0ESm3aS4RZmIiCiHGFRG4LoKPaEovMbI61MSdCHRa3HnDxERUa4wqIwgbDsIR51Rd/wkGDqDChERUS4xqIygL+IgYrsZBxWPJmFFY88hIiKi48egMoJQxIFSsRL5mfDoEhHH5YJaIiKiHGFQGYE/FIWuZRZSAECX8cMJOaJCRESUEwwqw3BcBX9o9EJvA8VGXhhUiIiIcoVBZRihqIOwbY9a6G0wKST6uKCWiIgoJxhUhmE7LmwHMLSxdZGpSfSEouPUKiIioqmFQSXHPLpEKOpwQS0REVEOMKjkmKlLWLaLMM/8ISIiOm4MKjmmaxKOq2BFOaJCRER0vBhUxoMCR1SIiIhygEFlHBiaRHcoku9mEBERFT0GlXHg0SWCYQeuq/LdFCIioqLGoDIOTF3CchyEufOHiIjouOQ1qMydOxdCiCG3tWvX5rNZx82jS0S484eIiOi46fn85lu3boXj9I867Nq1CxdeeCGuuOKKPLbq+EkhoACEufOHiIjouOQ1qNTW1qZ8vWHDBsyfPx/nnXde2usty4JlWcmv/X7/uLbveEgIBFlKn4iI6LgUzBqVSCSCX/ziF7jmmmvih/sN1dTUhMrKyuStsbFxgluZOVNnKX0iIqLjVTBB5fHHH0d3dze+8IUvDHvN+vXr0dPTk7w1NzdPXAPHyKNLhKMOIjxJmYiIKGt5nfoZ6Kc//SkuuugizJw5c9hrTNOEaZoT2KrseXSJYMhG2Hbg0QsmDxIRERWVgggqhw4dwjPPPIPf/OY3+W7KiJRSw05LDWZoErajEI46qPAa49wyIiKiyakgftW///77UVdXh4svvjjfTRnWr7e/gyt//DLebBnLAl6FcIRTP0RERNnKe1BxXRf3338/1qxZA10viAGetJ5/qw3hqItfb38n4+cYUoM/zFL6RERE2cp7UHnmmWdw+PBhXHPNNfluyrCijovmrhAAYOs/OtHRa43yjBiPLhFgKX0iIqKs5T2ofPjDH4ZSCieffHK+mzKsQx19cOJhw1XA03taM3qeqUtEbAfWKDt/lFKsuUJERJRG3oNKMTjQ3gsAMLTYQtqn3mhNBpeRxErpq1Er1Hb1RfFWawC2w/UsREREAzGoZODAsSAA4MOnNqDM1NHea+G1w12jPk8KAVcphEYJKp1BC72WDZtTRERERCkYVDJwoD0WVBY2lONDC+sAAE++0TLkune6+vCbv72TMoIihUBfZPhpHdtxccwfgeMouIpBhYiIaKDC3WZTIFyl8I94UJlXU4r5tWX43etHkotqq8tiBegOdQSx/jc7EbBslJo6PrKoAUBs+scfHr6Ufk8oil7LhtTAERUiIqJBOKIyipaeMEJRBx5NYta0EjROL8GimRVwFfDn3bFFtUe6Q7j9t7sQiC+IPdoTSj7f1CV6w86woypdfRE4yoVywd1BREREgzCojCKxPmVOdQk0GVtM+9H4aMmfd7eixR/GbY/vQldfFPGHcSzQv325xKOhL2qjzR8e8tpRx0Wb30KZacBVKqMFukRERFMJg8ooDsanfU6sLUved9b8GpR7Y4tqb3rkNbT3Wjihyod/PfdEAKlBRQiBco+Bd7rDQ3b/+ENRBCM2ykwdCoDDNSpEREQpGFRGkdiaPL+2NHmfR5c4P76oNmg5qK8wcfcnFmNBXTkA4NiggnDlXh294aGjKl19EbgKyZEal7uTiYiIUjCojCIx9TOvpjTl/o8umgFDE6gu9eA/P7EE1WUm6spjC2s7g5GUmihCCJSZOt7pCsGyY6MqiWmfUk//emaOqBAREaXirp8R9ISi6A7F1p7MrU4NKidM82Hj6tNRZuooNWPdWFliQJcCtqvQGYygrsKbvL7cq6OlJ4RjAQuzppWgJz7tU1sWv0YBjsOgQkRENBBHVEZwuLMPADCzygevoQ15vL7CmwwpQKxmSm18VKUtkDr9I4WAzxMbVYnYLrqCEagB0z5SCNic+yEiIkrBoDKC5nhQObGmbJQr+9XG66oMXqcCABVeA919EbT0hHEskDrtI0VsOoiIiIj6MaiM4HBXPKjUlo5yZb+a+IjKscDQoKJJAZ+h40hPCL3xwnAJUgoGFSIiokEYVEbQP6KSeVCpHSGoAEClz0BXXwRA/7QPEJv6iXKNChERUQouph1G0LKT60wG1lAZzUhTP0AsnFSXmNA1kXI/p36IiIiGYlAZxlutAQBAdakHlT4j4+eNNqICAD7P0IW5Ugg4TqyMvpQizbOIiIimHk79DGNvSyyojGV9CoBkLZVjAQtqDHVRNCngQLGWChER0QAMKsN4Mx5UBhd6G01NfOonFHUQjDijXN1PCgHX5Xk/REREAzGoDGNvfOpnLFuTAcBraKjwxmbURpr+GUzK2LSPyxEVIiKiJAaVNKKOi/1tsTN+xjqiAgxcpzL0xOThSBGf+uGIChERURKDShr723oRdRR8hob6CnPMz89kQe1gUggolwcTEhERDcRdP2nYjsKqE6th2S6EGPsOnNG2KKejSQHH5WJaIiKigRhU0lgyqxIbP3cath7syur52YyoAAAEeN4PERHRAJz6GQe15bETkccaVAQ49UNERDQQg8o4SNZSGcPUDwAogFM/REREAzCojIPEGpXOYAT2WMriq9gWZSIiIophUBkHlSUGdCngqlhYyZQQGFuwISIimuQYVMaBFCK5oLZtjFuUeTAhERFRPwaVcZLNFmUpBCIMKkREREkMKuOkJostypoUsDM/HoiIiGjSY1AZJ9lVpwWi3J9MRESUlPeg8u677+Jzn/scqqur4fP5sGTJEmzbti3fzTpu2U792LaC4hZlIiIiAHmuTNvV1YWzzz4bH/zgB/GnP/0JtbW12LdvH6ZNm5bPZuVEXTYjKlLAUbGDCXVt7KX7iYiIJpu8BpX/+q//QmNjI+6///7kffPmzRv2esuyYFn9H/x+v39c23c8Bk79KKUyOjNICiDqxM774dkGREREeZ76+d3vfocVK1bgiiuuQF1dHZYvX46f/OQnw17f1NSEysrK5K2xsXECWzs2NfGpn1DUQTCS2QpZKQRcpVhGn4iIKC6vQeXAgQPYuHEjTjrpJDz11FP44he/iHXr1uFnP/tZ2uvXr1+Pnp6e5K25uXmCW5w5r6GhwhsbF8l0+keTsaDCgwmJiIhi8jrD4LouVqxYgW9961sAgOXLl2PXrl344Q9/iDVr1gy53jRNmKY50c3MWm25CX/YxrFAGPNqSke9XgoB1+XBhERERAl5HVGZMWMGTj311JT7TjnlFBw+fDhPLcqtsW5R1qSA4yoeTEhERBSX16By9tlnY+/evSn3vfXWW5gzZ06eWpRb2WxRhgAcHkxIREQEIM9B5eabb8aWLVvwrW99C/v378eDDz6IH//4x1i7dm0+m5UzdeVeAGPboiwAuBxRISIiApDnoHLGGWfgsccew0MPPYTFixfjrrvuwne/+12sXr06n806LscCFgLhKIDsqtMqKI6oEBERxeW9XMfHP/5xfPzjH893M3JCKQXLceAohXKvkQwqrWMIKgCnfoiIiBLyXkJ/Mok6Cl5dwo0HjZlVPgBAZzCC3rCd0WsICAYVIiKiOAaVHLJsB6auxRaaACgz9eSoysGOYEavIYVA1OH+ZCIiIoBBJacitguPntql86pj9VP+0Z55UIkwqBAREQFgUMkpy3Fh6hKa6J++SRR6y3xEBbAdTv0QEREBDCo5plBq6tAkhgaVTEdUJKd+iIiIEhhUcsR2XGhSotxrQJMyeV7P3PjUz+GOvowWycbWqCgo1lIhIiJiUMkVy45N+5R7dRhSJkNJQ6UXpi4RcVwc6QmN+jqJgwm584eIiIhBJWcitgufocFraDB0kVxnokmRHFXJZEGtFLHKtDzvh4iIiEElZyzbRVWJAQDwGhL2gBGRuWNYpyLjC3F5gjIRERGDSs64SqHEEyv06zP0lKmbsSyoTU79cESFiIiIQSUXHFdBCgGfoQEAPLpMOVhwbnUJAOAfGWxRlkLAdVlGn4iICGBQyYlYoTcBbzyoGJqEEP2PJ0ZU2nsjyQMLh5NYo+IyqBARETGo5ELEcWEaGrxGrDsNTWBgzCjx6KiviJfSH2X6RwgBpVTKGhciIqKpikFlGFIK6BoQjjqjXmtFHVT4dIj4MIquSWhSZL1ORQiRMnVEREQ0VTGoDKPCa6Bxegm6+qKjrhexXYVy00h+bWgC+qAKs4ktypkEFQXWUSEiIgIYVEbUOL0ENWUetPdaw16jlIIQgM+jJe8zNAl9mBGVTBbUAlxMS0REBDCojMjUNZxYWwZNAkHLTntNxHFhaP0LaQFAlwK6llpLJRFUDndmUEpfceqHiIgIYFAZ1fRSD+ZWl6I7FEkbMCK2C1PXkluTgdgaE6+uwR4w9VNf4YXP0BB1FN7p6hvxe0opELFZ8Y2IiIhBJQOzppegvsKbdgrIst34icki5X7TkCnBRgqBOfF6KqOtU5ECPEGZiIgIDCoZMTSJE2vKYGgCXcFIysnGEcdFpU8f8pwSjzakumz/OpWRR1S0+AnKREREUx2DSoYqSwy8p6ECUhN4tyeUsm05UTp/IF2TQOogS8ZblOWgHUNERERT1dBPWBpWQ6UXFT4dhzqCeLcrDH8oCk0KmAPWpyQYUmDweth5GZ6iLDmiQkREBIAjKmNW4tGxsKECSxsrUe7TUWbqKQtpEwxNQgAp00Sz42tUOvsi6AkNX0pfixd84xZlIiKa6hhUsiCEQF25F8saq3BKQwU8+tBu1DUBTRMpW5RLPDpmVHoBAK8e7Bh2C7KUYFAhIiICg8pxMXUNlSVG2sfSFX0DgPm1ZQCAe5/dj2t/tg3/+9JB7G/rTRl5iZ2gzKBCRETEoDJOdCmgSwl70FqTNavm4kPvqYPP0NDea+Gx197FzY/uQNOf3kxeo8n41A+LvhER0RTHxbTjRNckDE3Aiqbu3mmo9OLmC0/GWtvF9kOdeH5fO17a344tBzoQjjrwGhqkEHAUy+gTERFxRGUceQ1t2LDh0SVWza/BrR9diGklBhT6zwGSAnBdBZdBhYiIpjgGlXHkMzREMwgbg09WFkJAceqHiIiIQWU8mbqW0ahIuoq1Ij6qQkRENJUxqIwjQxdDqtOmk65irULq1mYiIqKpKK9B5Rvf+AaEECm3hQsX5rNJOaVLiSHladNIjqi0B/trqygM2TFEREQ01eR918+iRYvwzDPPJL/W9bw3KWcMTUCTsVOUB5+uPNAJVT7oUiAUddAWsNBQ4YUuBYIRewJbS0REVHjyngp0XUdDQ0NG11qWBcuykl/7/f7xalZOGJqEJjFqUNE1idnTS3CgPYiD7UE0VHhhaBJBi0GFiIimtryvUdm3bx9mzpyJE088EatXr8bhw4eHvbapqQmVlZXJW2Nj4wS2dOx0TUDXJGx39JOQ59akHlhoaBKW7cLmKcpERDSF5TWorFy5Eg888ACefPJJbNy4EQcPHsS5556LQCCQ9vr169ejp6cneWtubp7gFo+NISV0MbQ6bTrzBm1R1jWBqOMiwqBCRERTWF6nfi666KLk35cuXYqVK1dizpw5ePTRR3HttdcOud40TZimOZFNPC5SChi6QJ81etjo36LcP6JiOwoR20WJZ1ybSUREVLDyPvUzUFVVFU4++WTs378/303JGZ9Hy2ibcWLq52hPGH0RG5oUcFQsqBAREU1VBRVUent78fbbb2PGjBn5bkrO+EYooz9Qpc/A9NLY0MmhROE3BVgMKkRENIXlNaj8+7//O55//nn84x//wF//+ldcfvnl0DQNV111VT6blVOGJvtro4wiUUo/eeaPFAhHnXFrGxERUaHL6xqVd955B1dddRU6OjpQW1uLc845B1u2bEFtbW0+m5VTHl1CZFCdFoitU/nb4a7kglpDCgQtBhUiIpq68hpUHn744Xx++wmhS4FM68sOLqVvaBKhiAPXVZAj1GEhIiKarApqjcpkZOgSmhQZ1UOZW10CIDb14yoFQ5OIuA63KBMR0ZSVk6Di9/vx+OOPY8+ePbl4uUnF1CU8mswobMyaVgJDEwhHXbT6wzA0gaijuKCWiIimrKyCypVXXon77rsPABAKhbBixQpceeWVWLp0Kf7v//4vpw0sdh5NwtRlRtuMNSkwe3psVOVgexC6JuE4LrcoExHRlJVVUHnhhRdw7rnnAgAee+wxKKXQ3d2Ne++9F//5n/+Z0wYWOyEEyr16xtM3g9epQIBTP0RENGVlFVR6enowffp0AMCTTz6JT33qUygpKcHFF1+Mffv25bSBk0GZaWRURh8YukVZQMDiFmUiIpqisgoqjY2NePnllxEMBvHkk0/iwx/+MACgq6sLXq83pw2cDExDQojMgkq6nT+9PEWZiIimqKy2J990001YvXo1ysrKMGfOHHzgAx8AEJsSWrJkSS7bNymYuoQUEo6roI2yzTgxotLqt9AXsWFoEn0RB0opiEwLshAREU0SWQWV66+/HmeeeSaam5tx4YUXQsrYwMyJJ57INSppmLoW38HjQpPaiNdW+AxUl3rQEYzgYHsQ82pKEYmfomzqIz+XiIhossm64NuKFSuwYsUKAIDjONi5cyfOOussTJs2LWeNmyxiW5Q1RGwXXmP0sDGvphQdwQgOdfTh5PpyBCM2IjaDChERTT1ZrVG56aab8NOf/hRALKScd955OO2009DY2IjNmzfnsn2TgpQCJaaGaIa7d2ZW+QAAbYEwdClgOzxFmYiIpqasgsqvf/1rLFu2DADw+9//HgcPHsSbb76Jm2++GbfddltOGzhZVIxhi3JNWewU5WMBC0LESvBzizIREU1FWQWV9vZ2NDQ0AACeeOIJXHHFFTj55JNxzTXXYOfOnTlt4GTh9WgZn/lTWx7bOXUsYCXv44gKERFNRVkFlfr6euzevRuO4+DJJ5/EhRdeCADo6+uDpnEdRTqmrkEAcNXocaW2zAQAHOuNAIgdbBjkFmUiIpqCslpMe/XVV+PKK6/EjBkzIITABRdcAAB45ZVXsHDhwpw2cLJInPkTzWD3TmLqpzNowXFjhxMGIyz6RkREU09WQeUb3/gGFi9ejObmZlxxxRUwzdgIgKZpuPXWW3PawMnC1CUMXSJqK5ij9Pq0Uk9sEa2r0BmMwOfRYNkuoo4LQ+OB10RENHVkvT35n/7pn4bct2bNmuNqzGSmaxI+j0QgNPrIiBQC00s9aAtYONZrYX5tKaywg4jNoEJERFNL1p96zz//PC655BIsWLAACxYswKWXXoq//OUvuWzbpFNuGhnv3qktj41StQcsGJpElFuUiYhoCsoqqPziF7/ABRdcgJKSEqxbtw7r1q2Dz+fD+eefjwcffDDXbZw0fB4to8W0QH9QOdZrQQoBBZdblImIaMrJaurn7rvvxj333IObb745ed+6devwne98B3fddRc++9nP5qyBk4nX0IBMg0pi5098i7ICtygTEdHUk9WIyoEDB3DJJZcMuf/SSy/FwYMHj7tRk1VyQW0GIyPJqZ/eWFDRheQWZSIimnKyCiqNjY3YtGnTkPufeeYZNDY2HnejJitT1+DRZEYjIzWDRlRiW5QZVIiIaGrJaurny1/+MtatW4cdO3bgrLPOAgC89NJLeOCBB/C9730vpw2cTDy6hGloCEUclI5ybX/Rt/6gYkVd2I4LnTt/iIhoisgqqHzxi19EQ0MDvv3tb+PRRx8FAJxyyil45JFHcNlll+W0gZNNmanD3xcd9bqa+NRPIGwjHHWgawJ9EQdRR4GHKBMR0VSRdR2Vyy+/HJdffnku2zIllJk6bDX61E+pR4PP0BCKOmjvtTCj0pfcouzzMKkQEdHUwDmECWbqEoAY9TohRP8W5YAFTQo4LrcoExHR1JLxiMq0adMgxOgfsADQ2dmZdYMmO1PXoAkBx1XQ5Mj9WVNm4nBnX3LnDwQYVIiIaErJOKh897vfHcdmTB2mIWHoApbtoMQzcvcPHFEBAAGBSJSHExIR0dSRcVDJ5hyfDRs24LrrrkNVVdWYnztZmbqEqWuIOqMXfquNn6Kc2PmjS25RJiKiqWVc16h861vf4jTQIEIIlJlaRrVU+ou+RQAAhiYQinLqh4iIpo5xDSoqw3LxU02ZqSPqZhBUBhV90zUJy3YzqmxLREQ0GRTUrp8NGzZACIGbbrop300ZVz6PntGZPzUDDiZUSsHQBGzb5Zk/REQ0ZRRMUNm6dSt+9KMfYenSpfluyrgzdQlNSjju8GGl17JRZsaWEEVsF/6wDV1K2K7iiAoREU0ZBRFUent7sXr1avzkJz/BtGnT8t2ccWfqGgxNjBg4ukMRBCMOqkoMALHDCTUp4CjFERUiIpoyCiKorF27FhdffDEuuOCCEa+zLAt+vz/lVoxMXcKja7CGCRyOq2BICddVQ9apAKylQkREU8e4BpVzzz0XPp9vxGsefvhh/O1vf0NTU9Oor9fU1ITKysrkrVhPapZSoNTUhh1RsWwHHl0CYugpyhICYdZSISKiKSLrs35c18X+/fvR1tYGd9AOlve///0AgCeeeGLE12hubsaNN96Ip59+Gl6vd9TvuX79etxyyy3Jr/1+f9GGlQqvjqM94bSPWbYL05AIRewBW5QTO39ihxMSERFNBVkFlS1btuCzn/0sDh06NGQLshACjpPZB+n27dvR1taG0047LXmf4zh44YUXcN9998GyLGha/wF8pmnCNM1smlxwvIY+7Ik/lu2iwmdAkxI1g4q+GZpEX8SBUirjIw2IiIiKVVZB5brrrsOKFSvwxz/+ETNmzMj6A/P888/Hzp07U+67+uqrsXDhQnzta19LCSmTjdeQEEh/5o+rXJR6NHRLgWklsaDSHkgEFYGI4yLqKHh0BhUiIprcsgoq+/btw69//WssWLDguL55eXk5Fi9enHJfaWkpqqurh9w/2Zi6Bo8e2/mjyf5A5ioFKQTKvQY0LYzppUPL6PdFbEQcN7aOhYiIaBLL6pNu5cqV2L9/f67bMqWYuoRHG7rzJ2K7MDSJcq8OXUpU+GLbkzuDETiugq4JRB1uUSYioqkhqxGVL33pS/jyl7+MlpYWLFmyBIZhpDx+PEXbNm/enPVzi4mUAqVeDR3xc3wSLNuF19DgMzSYhkCZqUOXArar0BG0UFfuBcCib0RENDVkFVQ+9alPAQCuueaa5H1CiOQCz0wX00515aaOlkE7fyzbwfRSL6QU8Bk6ulQUNWUmWvxhHAvEgooCOKJCRERTQlZB5eDBg7lux5Tk8wztfttRKPfGRqi8eqzMfk2ZBy3+cPIUZSkka6kQEdGUkFVQmTNnTq7bMSWZeurOn9hWbwWvEVtc6zE0KCBZSyVR9M2QAkGLQYWIiCa/rAu+AcDu3btx+PBhRCKp6ywuvfTS42rUVOE1Unf+xLYcS/g8saBixLctJ6rT9hd9kwhFHbiugpTcokxERJNXVkHlwIEDuPzyy7Fz587k2hQAyXoqXKOSmcTOn0h8AW2sdL4Gb3zbsa5JCGDIeT+GJmDZLiKOC6+cvLVmiIiIstqefOONN2LevHloa2tDSUkJ3njjDbzwwgtYsWLFlNm1kwtSCpSYWvKQwYjtxnb5aLEfi6EJ6JrAtHgtlfYB1WmjrsudP0RENOllFVRefvll3HnnnaipqYGUElJKnHPOOWhqasK6dety3cZJrcKrJwOH5bio9PUPchmahDagOm1bIFH0TcBhLRUiIpoCsgoqjuOgvLwcAFBTU4MjR44AiC2y3bt3b+5aNwV4PbEFszEqZSeQoUnoUmJ6PKj0Wjb6InZsug1IjsQQERFNVlmtUVm8eDFef/11zJs3DytXrsQ999wDj8eDH//4xzjxxBNz3cZJzatrkEgsqJXJHT8AoEkBT3yLcrlXRyBso81vYW6NDiggaqsRXpmIiKj4ZTWi8h//8R9w3dhv83feeScOHjyIc889F0888QTuvffenDZwsjMNCUMXCIRteHUJn5G6ONaMB5X6ci8AoDUQKxCnS4FgxJ7w9hIREU2krEZUPvKRjyT/vmDBArz55pvo7OzEtGnTsj5JeaoydQ2mpqEjaKGh0jvkoMESj4ZWV6G+wsT+Y71o9Q/cojx6UAlHHUgheIAhEREVpeP69Nq/fz+eeuophEIhTJ8+PVdtmlK0+M6fqKNQVWIMedzUNbjKRV1FfETFHxtRMTSBcMSF4w4//eO6CntbA8nnEBERFZusgkpHRwfOP/98nHzyyfjYxz6Go0ePAgCuvfZafPnLX85pA6eCcq8OQxcoNYcOcBmaBCBQH69O25ac+hl9i3JXXwTH/Ba3MRMRUdHKKqjcfPPNMAwDhw8fRklJSfL+T3/603jyySdz1ripwufRUGbq8OpDi7fpWmwqrT45otJf9C3qKFjDbFFWSuHd7hB6LXvYa4iIiApdVmtU/vznP+Opp57CrFmzUu4/6aSTcOjQoZw0bCox9VhQSZTOH8iQEpoQyTL6bfFpHF2TcJzhR1S6+6JoC1jwGRpHVIiIqGhlNaISDAZTRlISOjs7YZrmcTdqqqnw6pg9vSRla3KCoQvoUmB6aWz9SjDioDccX0QrBEKRoccVJEZTlFLweTQWhiMioqKVVVA599xz8fOf/zz5tRACruvinnvuwQc/+MGcNW6q0DWZXCw75DEpoWsCmpSo8sXCSmKLss/QcKC9N1laP6EnFEVrIIwqnwdSxKaIEucxERERFZOspn7uuecenH/++di2bRsikQi++tWv4o033kBnZydeeumlXLdxSjPiIcV2FeorvOgORdHmD2N+bRmmlXjQ0Wthz1E/Fs2sxPT4mUBHekJwHAWvocF2FVyl4Lgqud6FiIioWGQ1orJ48WLs3bsX55xzDi677DIEg0F88pOfxGuvvYb58+fnuo1TmhACPkOD7bioq4hNqyUW1AJAdZmJqK2w+6gfXcFIbDSlJzaaAgCaEHBcBXuEbcxERESFKqsRFQDwer248MILsWzZsmSV2q1btwIALr300ty0jgDEqtc6rkLdoOq0CbXlJtr8Yew56kelz0DEVqgpi613kTJWT8Xl1A8RERWhrILKk08+iX/+539GZ2fnkLUPQgg4ztAFnpS9Eo8GR8Wq0wJIW8CtttxEW8CCPxxNnrYMxEZUbMURFSIiKk5ZTf186UtfwpVXXokjR47Add2UG0NK7umahEL/eT9tfmvINUII1JWbqCkzUTLgBGZNCigXcBwGFSIiKj5ZBZXW1lbccsstqK+vz3V7KA1DCgCiv+hbIJx2F48QAuagonFCCLjK5YgKEREVpayCyj/90z9h8+bNOW4KDcfQJCQEqstiUzrhqAt/OPOTk2NhhUGFiIiKT1ZrVO677z5cccUV+Mtf/oIlS5bAMFIP01u3bl1OGkcxuiagabFpnOmlHnQGI2j1h1HpG3qIYVoCHFEhIqKilFVQeeihh/DnP/8ZXq8XmzdvhhD99TmEEAwqOWZoEpoUsB2F+nITncEI2gIWTq4vz+j5XKNCRETFKqugctttt+Gb3/wmbr31VkiZ1ewRjYGhSRhSwnZd1FV4saclkDzzJxNSCNguy+gTEVHxySplRCIRfPrTn2ZImSCaFDD0/uq0ANAylqAiwROUiYioKGWVNNasWYNHHnkk122hEfiSRd/ipygHhm5RHo4mBA8mJCKiopTV1I/jOLjnnnvw1FNPYenSpUMW037nO9/JSeOon9fQYDsKDYktymMaURGIOgwqRERUfLIKKjt37sTy5csBALt27Up5bODCWsodU9fgov+8nza/BaVURv2tDThBmT8fIiIqJlkFleeeey4n33zjxo3YuHEj/vGPfwAAFi1ahDvuuAMXXXRRTl5/MjE0CQGBmjITAkDEcdHdF8W0Us+oz5VSwOEJykREVITyuhp21qxZ2LBhA7Zv345t27bhQx/6EC677DK88cYb+WxWQdI1AYVYYKkui5/5E8hs+id2gjKr0xIRUfHJa1C55JJL8LGPfQwnnXQSTj75ZNx9990oKyvDli1b8tmsgpSopeK4Aw8nzGxBbfK8HwYVIiIqMllN/YwHx3Hwq1/9CsFgEKtWrUp7jWVZsKz+D2e/3z9Rzcs7jyZhxBfF1pd78Qb8GddSkQI8QZmIiIpS3guh7Ny5E2VlZTBNE9dddx0ee+wxnHrqqWmvbWpqQmVlZfLW2Ng4wa3NH0MT0DUJ21HJBbWtGW5R1qSA6yq4DCpERFRk8h5U3vOe92DHjh145ZVX8MUvfhFr1qzB7t270167fv169PT0JG/Nzc0T3Nr80TUJU5eIum7/KcoZjqgkdvpwRIWIiIpN3qd+PB4PFixYAAA4/fTTsXXrVnzve9/Dj370oyHXmqYJ0zQnuokFo8SjoScURX2i6NsYaqkAXKNCRETFJ+8jKoO5rpuyDoX6+QwtVp02PqLSFrDgqlj4sGwHj732Dl7c3572uQqAoxhUiIiouOR1RGX9+vW46KKLMHv2bAQCATz44IPYvHkznnrqqXw2q2CZhgYFhZoyb2yBrKvQFYygxR/G95/dj3e7Q/BoEqtOrIYmh9ZL4QnKRERUbPIaVNra2vD5z38eR48eRWVlJZYuXYqnnnoKF154YT6bVbAMTQIqtji2psxEW8DCvc/uw98OdyeviTguOoMR1JanTpFJCEQcZ4JbTEREdHzyGlR++tOf5vPbFx2PLqHJ2OGEDRVetAWsZEi58NR6vHa4G+29Flr94aFBRQIRmyMqRERUXApujQoNz6NJGFqslsrs6hIAQH2Fif+8bDHWfegknFCVWLsydJFt7LwfHkxIRETFJe+7fihzA2uprF45B4tmVuL02dPg82gAEF9k25O2Yq2UAhEGFSIiKjIcUSkiA2uplJk6zllQkwwpAEasr6IJAduOnaBMRERULBhUikyJRxt2CidZXyVNxVpNCjhgGX0iIiouDCpFJlFLJZ26EUZUZPwEZRZ9IyKiYsKgUmRMQ4OL9GEjMaLS3msNCSSJ834YVIiIqJgwqBQZQ5MQw2SNaaUe6FLAVbGwMpAmBRzF836IiKi4MKgUmYG1VAaTQqBumHOApABHVIiIqOgwqBSZgbVU0unf+ZM6oiKEiJ33w6BCRERFhEGlyAyspZJOckFtmqJvAgwqRERUXBhUiszAWirpJLcopyn6pgDYwzyPiIioEDGoFKERa6mMMKICAMwpRERUTBhUitDItVRiIyppy+gLnqBMRETFhUGlCI1cSyU2otIZtIaMukgBRGwOqRARUfFgUClCI9VSqSox4NFk+loqQiA6zCJcIiKiQsSgUoRGqqUihEhO/wxeUKvxBGUiIioyDCpFaLhaKkop9Fr2sAtqZfwEZZdblImIqEgwqBSh4Wqp9ISi6Oi1UFPmATB0QW2sjL4LRzGoEBFRcWBQKULD1VIJRmyU+3TUlKYvox8LKiyjT0RExYNBpUgNrqUStGz4PBp0KVETL/rWGkgdUZEidoIyDyYkIqJiwaBSpAbXUukJR1Ff7oUmBaaXJqZ+0oyouCyjT0RExYNBpUiZhpaspBKxXWhSoL7CC6+uJYNKZzCSMuoiBeBy6oeIiIoIg0qRMjQJxBfF9oSiqCn1xGqo6AI+Q8JrxH60xwZM/yROUOZ5P0REVCwYVIpUopZK1HERdV3MqPJBCAFT16CUQF28Qm2Lf+iZP8wpRERULBhUilSilkpnMIJKr5Gc7vHoEi5c1A13irLiiAoRERUPBpUilailEoramFHljU0FAdA1AUCgIV70rW3wKcqCi2mJiKh4MKgUqUQtlQqfgZoyM3m/JgUEBp6iPLQ67eCKtkRERIVKz3cDKHslpoYyU0ep2f9j1GUseyamfoZUpxWCJygTEVHRYFApYrOnl0KXIuU+XROQUiRHWQZP/UgBRGxO/RARUXFgUCliZebQH58uBfQBRd+6+qKwbAemrgFInKDsTGg7iYiIssU1KpOMLiU0KVDi0eAzYuGkbUAtFU0KOA54gjIRERWFvAaVpqYmnHHGGSgvL0ddXR0+8YlPYO/evflsUtHTpYAuJFwF1FcM3aIsRewE5bDNURUiIip8eQ0qzz//PNauXYstW7bg6aefRjQaxYc//GEEg8F8NquoSSmg6wK2q1Af36I8cOeP19DQa9l444gfXcFIvppJRESUkbyuUXnyySdTvn7ggQdQV1eH7du34/3vf3+eWlX8TF0iGLYxe3oJXjnYid1H/fjYkhkAYlM/Myp9OBaw8Pd3ujG/rgwzK32QgxblEhERFYKCWqPS09MDAJg+fXraxy3Lgt/vT7nRUKYuYbsKK+bG+nHboU7YKYcTxg4w1KXE7iN+7G31w+JUEBERFaCCCSqu6+Kmm27C2WefjcWLF6e9pqmpCZWVlclbY2PjBLeyOHgNDa5SeE99OSq8OoKWg91Hh4a6Cl+s9P7B9j4c6QrloaVEREQjK5igsnbtWuzatQsPP/zwsNesX78ePT09yVtzc/MEtrB4aDJ2SrImBc6Ij6q8crAz7bWmrqHUo6MjGIFS3AlERESFpSCCyg033IA//OEPeO655zBr1qxhrzNNExUVFSk3GsrQJBIrTlbOSwSVjmGDiM/QELRshKOsWEtERIUlr0FFKYUbbrgBjz32GJ599lnMmzcvn82ZNBIjKgDw3sZpMDSBVr+Fw519aa/3GhKW7aLXsieukURERBnIa1BZu3YtfvGLX+DBBx9EeXk5Wlpa0NLSglCI6yWOhyElNCHguAo+j4Zls6oAAK8OM/0jhIBSQNCKTmAriYiIRpfXoLJx40b09PTgAx/4AGbMmJG8PfLII/lsVtHTNAFNAm58qmflvGoAw69TAWI7hTr7WFeFiIgKS17rqHDx5vjQZexgQttRMDTgjLnTAABvtQbQFYxgWvwcoIF8Hg2BsI1w1IE3XnqfiIgo3wpiMS3lli4FNCnhxINgdZmJk+rKoABsPZR+VMVraLCiXKdCRESFhUFlEtI1CV3G1qgkJHf/HEgfVKQQUFDos1j4jYiICgeDyiRl6jIlqJwZX6eyo7kb4Wj6MGJqGjqCVtrHiIiI8oFBZZIaHFTmVpegrtxExHHx+jvdaZ8zcJ0KERFRIWBQmaQ8g4KKEGJA8bcR1qnYDoJcp0JERAWCQWWSMnUtuT05IbFNecvbHejoHTrFI4WA6wJ9EY6oEBFRYWBQmaQ0KZCsox+3aGYFGqeXIGDZ+OYfdqMvMnTkxKNLdAZZT4WIiAoDg8okpWsizX0Sd3z8VFSVGDjYHsSGP70J20k938dnaPCHo7BsjqoQEVH+MahMUrqMHUw4uKheQ4UXd1x8Kkxd4rXmbvzP5rdTrvEaGsIRF0FuUyYiogLAoDJJ6VqsOu3ABbUJJ9WX46sfWQgpgKf3tOLhrc3JxzQp4CjFBbVERFQQGFQmKV2KWNG3YY4pOHPedFx33nwAwIOvHsarBzuSj5lcp0JERAWCQWWS0qWMjY6kGVFJuGjxDFy8ZAYA4IldLcn7vYaGQDiKiO0O91QiIqIJwaAySelSQBdyxKACABcvjQWVHc3d8IeiAGIjKpbjIswFtURElGcMKpOUlAK6LmCPElQap5VgbnUJHFdhS3z6x9AkorYLK8oRFSIiyi8GlUnM1CXcUYIKAJx7Ui0A4C/72pP3CQiW0iciorxjUJnETF2OOqICAOcsqAEA/P2dbvTEp380KdIWhCMiIppIDCqTmNcYWkY/nZlVPsyvLYWrgL++HRtV8WgSgTCDChER5ReDyiSmyaHVaYdzzoLY9M+L++NBRZcIRx3u/CEiorxiUJnEdJn5j/eck2LTP7ve7UFXXwQeXSLiuCylT0REecWgMonpmhhSQh9A2umghgovTq4vi0//dECXAlHbRZg7f4iIKI8YVCYxXQpoMrWWSiAcxeHOvrRTOolFtX/ZdwxCCEAIjqgQEVFeMahMYromIWX/CIrtuPCHo5hR6U3u7hno7HhQ2X3Ej45eC1IIhCIMKkRElD8MKpNYbESlv4z+sV4LMyp9aKj0Iuo6Q6aF6sq9WNhQDgXgpbc7YGoybaAhIiKaKAwqk1hi6sd2FfyhKLyGhnm1pagpM1Hq0RG0ho6WnBtfVPvi/nYYukTYdhF1uE6FiIjyg0FlEtPiJyhbUQe9ERvzakpR4TXgNTTUV3jht9JM/8yPBZU9R/0IRxxEbQcWtygTEVGeMKhMYkIIeHSB7lAUDZVezKzyJR+rKTOhCTFktKS6zMTc6hIAwJ4WPyKOYil9IiLKGwaVSc7UNVSXeXBiTWlKAbhKn4GqEiN5YvJAS2dVAQB2vtsDQHFEhYiI8oZBZZKr8hlYUFeGcq+Rcr+UAjOqfAjbQxfVLptVCQB4vbkbEhKhYc78sbl2hYiIxhmDyiR3wrQSzKj0pX1seokHJR4dfYO2IC+aWQkpgCM9YQTCUfjDQ0ddei0bu97t4bQQERGNKwaVKczn0VBb7kFg0KLaUlPHSXXlAIC3WnsRirhDRk/aA2G090Z4cCEREY0rBpUprrbcC2DoNM7S+PTPnhY/IoN2/kRsF0e6w+i1bATT7BwiIiLKlbwGlRdeeAGXXHIJZs6cCSEEHn/88Xw2Z0qq8hko9xpDRkaWxRfU7nq3BxHbTZni6QxGELBsVPkMtPdG0p4nRERElAt5DSrBYBDLli3DD37wg3w2Y0rTNYkTqnzos1ODysIZ5TA0gY5gBK0BKzmi4roKR3tCMKREiUdH0LJ5cCEREY0bPZ/f/KKLLsJFF12UzyYQgGmlHnh1DaGIA59HAxDb1nxKQwX+/m4P3jwawFnzqwEAPaEoOoIRTPN5YGgC3SEXvZadfB4REVEuFdUaFcuy4Pf7U250/MpMHdNLPUN29yTWqbzVGkBvODb1cyxgwXEVPLqEEAJKgetUiIho3BRVUGlqakJlZWXy1tjYmO8mTRoNFV44rps8wBDoX6fyZksAvVYUgXAULf4wKgbUZPEaGtepEBHRuCmqoLJ+/Xr09PQkb83Nzflu0qRRVeJBuddAr9W/VmVBXRl8hoZey8bbbUEc7QmjL+KgdMA0j8/QEIxwnQoREY2PogoqpmmioqIi5Ua54dEl6iu8CEb6p3F0TWLRzFgf7zzSg85gBCWGBiH6S/F7DQkr6qYEHCIiolwpqqBC46umzISuSUQG1ExJTv8c9cMfiqLCl1qKn+tUiIhoPOV1109vby/279+f/PrgwYPYsWMHpk+fjtmzZ+exZVNThU/H9BIPuvuiqCkzAQDLGmMLave39UIIpBxsmJBYpzKnWqWMthARER2vvI6obNu2DcuXL8fy5csBALfccguWL1+OO+64I5/NmrKEEKiv8MIacFDhnOpSVHh1hG0XbX4r7fO4ToWIiMZLXkdUPvCBD3C3SIGZXupBqakjGHFQZuqQQmDJrCq8tL8dtz2+C/NqSrFsViWWzarC4hMq4TU0eA3JeipERDQuuEaFUngNDbVlJgIDaqp8cvkJmFtdAgA42B7E4zuO4Jt/2I11D78G23G5ToWIiMZNXkdUqDDVlpt4pyuErmAEFT4DJ9eX4/tXnYauvgj+/k4PXn+nGy+8dQxHe8J444gfyxqruE6FiIjGBUdUaIhpJR4snFEO05BoC4TR5g/Dsh1MK/HgvJNrse5DJ+H9J9UCAF4+0AGA61SIiGh8MKjQEFIKzJpWgtPnTMN7Z1ehptxETziKjt7+xbTvOzF29s+WAx1QSrGeChERjQsGFRqWrknUlXuxdFYlFjZUwHJcuPHFz+9trILP0NARjGBfW29ynUpvmOtUiIgodxhUaFRCCFSXelDq0dBnxQ4n9OgSp82ZBiA2qgIAPo+G1vihhURERLnAoEIZ8RoaaspM9A4osb8qPv2TWKdSZurotWz0hjn9Q0REucGgQhmrKY9Vq02MmKyYMw26FHinK4Tmrj4YmoTtKPg5/UNERDnCoEIZq/QZKDf7T1guNXUsjZ8FtOXt/t0/rf4wC/kREVFOMKhQxgxNoq7CTDlhefD0T6mpIRCOcvcPERHlBIMKjcn0Ug80KRF1YvVSVp44HQLAvrZetPdaMHUNEVuhJ8TpHyIiOn4MKjQmFV4DlT4DgfiC2VhxuAoA/bt/PLpER2+E0z9ERHTcGFRoTKQUaKjwImT3T+2sOnE6gAHTPx4dXX0R9EWcvLSRiIgmDwYVGrOqEgNeTUM4Ggsiq06sAQDsercH/lAUPo8GK+pw9w8RER03BhUaszJTx7RST3L6p6HSi7nVJXAVsPUfnQAAXUq0Dyi5T0RElA0GFRozIQTqKkxEHCe5DuWs+bFRleffOgYAKPPq6ApGk6MuRERE2WBQoaxMK/Gg1NQRjK9D+eB76gAAO5q70d5rwWdoCEUd+Ln7h4iIjgODCmXFa2hoqPAmg0hDpReLZlZAAXjuzTYIISCFQHuQ0z9ERJQ9BhXKWl25F4YuYNmxUZULFtYDADa92QalFMo8Oo4FLOx6txt7W/x4u60XhzqCrLFCREQZY1ChrFX4dFSXmujpiwWPsxZUw9Ql3u0OYW9rAKWmBqEE2vwRvNMZwoFjQbxxxI99rQHY8YJxREREI2FQoawJITCjygtHKTiuQolHx9nxRbWb9sSmf6aVelBbbqKuwouGSi9mVvrQGYygIxjJc+uJiKgYMKjQcZle4kFViZGczjn/lNii2r/sO5acEhpIkwKGJtHc2Zc8hZmIiGg4DCp0XHRNYmaVD6GoDaUUFp9QibpyE8GIg1cOdKZ9TpXPQGcwwjorREQ0KgYVOm41ZSbKTB29lg0pBD64MDaqsunN1rTX65qEoUm808VRFSIiGhmDCh03r6GhodKbLJl//sL+miod8VET23Hx0v52bHz+bTR39aHKZ6CjN5J8nIiIKB093w2gyaG23Ivmzj6Eow5mVPqwaGYF3jjix2OvvQuvoeHPu1vQFd8d9PLb7bjnU8ugS4nmrj5Ul5nQpMjzOyAiokLEERXKiQqvjtpyL7r6Yrt5EqMqv339CB7Z1oyuviiqSgzMqPSiqy+K//jtTiil0MlRFSIiGgGDCuWEEAKN00pgGhL+UBRnL6hBlc8AACydVYmvfXQh/nfNGdjwyaWYUelFq9/CnX/YjVDURXNXH0IRnglERERDCZU4Va4I+f1+VFZWoqenBxUVFfluDgFo7uzDG0d6UFfuRa9lI+q4qCv3plzT4g/ja7/+Ozr7Iji5rgzXf3A+aspMzKzyob7Ci1KTM5JERJPZWD6/OaJCOTWzyodZ00pwrDeMKp8xJKQAQEOFF3detgjlpo632nrxP5vfxt8Od2HPUT+2HerEvtYAOnot9ISiCFo2wlGHu4OIiKYojqhQzgUtGzuau2E7CtNLPcNet7clgP/47U6Eo7Fy+lUlBs5ZUIPT51RhRoUPuiahSUBKAY/UMKMqVuG2xMMRFyKiYjaWz++CCCo/+MEP8N///d9oaWnBsmXL8P3vfx9nnnnmqM9jUClcrf4w/v5OD6p8BryGBgBwlULEdiEE4NEkhBA42hPCn3a14Lk329A94LBCjy5RU+ZBdamJ6lIP6ipMNFR6cUpDBU6qL0N9hRflXiNfb4+IiI5DUQWVRx55BJ///Ofxwx/+ECtXrsR3v/td/OpXv8LevXtRV1c34nMZVAqXUgr7WnvxdnsvpBAQAAQEDF0ACvHy+gKmLmEaGjQB7HinB8/sbsW2Q50YaaanttzE/NpSnFRXhvc0VOCUGeU4ZUYFR1qIiIpEUQWVlStX4owzzsB9990HAHBdF42NjfjSl76EW2+9dcTnMqgUNst20NzZB10KeA09HkoklAL6Ig6CVhQ9IRsBy0bEdhC1XUAIuK5CIBRFVyiKrmAEnX1RvNPVh7eP9aLVP/xWZp+hwVUKSgGOUoACppUaaIgfiDij0oeqEgNSAFIIaFJAxP/UZf+frgJCUQehiINQ1EE46sTfgwZT12AaEqYu41/H/vRoEgqA4yq4SsF2FRL/1xKJoCYApZBsY+JPTQpIKaDF2xK79bdRE7F2ShF/LQFEHReW7cKKurBsB65S8OparE1GrE2aGFqbJnFXrEWA7bqwXYWo7SLqKriJhCj6//BoEh49djN1DbqWeHY/hdh7S3yV+Bk4bvzvbqxPbCf2/ex4P5la/+t69NhUHzDgvSZfW8XDq4Ljxtvt9L+OJgUMKWM/Qy31+TLeZ/F/ElBKJf903P6fmZv4eSF2vRDD/D3ZjwJpujgrI/1XOPE9ZfybDXwPyWuA5L+RxPtM/PtC8spYe2WaRo/0MTDw328m72Ok9iVeZ/C/oMTVg5sxsN8x4D0lvsfANg78XoNfIxtD/5WnUhi+LQPf52jff/B7HtgXg38q2byuGvIq/e8tk75JjIDXVQxdb3g8xvL5nddfQSORCLZv347169cn75NS4oILLsDLL7885HrLsmBZ/R9Ufr9/QtpJ2TF1DQvqytM+VmrqqC03AcQ+dMNRB+H4h27QshGKuIjGP4wSHyKuqxAI23i7vReH2vtwtCeM1kAYrT1hBOOhYrD23gjaeyPYdYT/VoiIsrFsViV+e8M5efv+eQ0q7e3tcBwH9fX1KffX19fjzTffHHJ9U1MTvvnNb05U82iCGPGzf9JsEIr/1qvgqP7fzM9TtXDd2G/sthMbCejsjaCrz4IuY6+V+K26PRjBu10hvNMVQqs/jN6wHXtdxH9jUQouVOz14r9ZC4GU3/I9moTjKkQdd8BNIeK4iNguoraLiBNbe6OJ/tERIUT/b10q9puNgICMj5bI5G+JgOvGRgxi71Ml73PiAS21vYAeP4Xao0kYWuxX7qgda2PEibXJTbzRRF+m+a1Vk6kjSmLIr1gKtqMQdRRsN/baQ3ZgKQADfkuODzYk32PyTymgSwkpEP9eiL1uvD9td8B7HdBv/aMY8RGSeFuliPWDECL5s3Pc/lGcwaMniRGFZPsQ+1kl2ph464n+SYywDPzNNjlSkHj9+M90vCT6IdGO/hExJPtcxRuTGEUR8RGpxHtMDEsN7Ne0LU4zTNY/UqYSP+ZhLh7wMoPaOPB1Ev8G0z87/b0D/93KASNZ6a4ePHaQbqAos5+ZGuGrfol/8aK/m1PeZ7rvn24UI83Y55Brk6+bZqQl0cbE5Sk/gyHfQSX/d7iBNDHw3xZi/z3Mp6Ka1F+/fj1uueWW5Nd+vx+NjY15bBGNNyFiQ/mj/UM9ocqX0esNHuIe+sE8dslgoVT/ehzR/9pqwHA1gOS0RKav3T+EO/Q9jPQ6mczqprti4CsOfA+J690BbRrJ4OmSga839D+bSI6cDXztgR/OA1833fTFwOcM/IBP3Jd47mhtHNimgc8d/Jr5kK4/Bj8OjD6kP9rrDH49YOyhLN3rZ9t/A6cvMpm2yNXPKd20yUDDtWWkaZfRDOzjXL3ucD+3dK8jhUhugCgUeQ0qNTU10DQNra2pp+y2traioaFhyPWmacI0zYlqHk1CuQgm6V5TE4A23O+JYvT55JFee+hzM32xXL7X8Rs1SBiu/4hoasvreI7H48Hpp5+OTZs2Je9zXRebNm3CqlWr8tgyIiIiKgR5n/q55ZZbsGbNGqxYsQJnnnkmvvvd7yIYDOLqq6/Od9OIiIgoz/IeVD796U/j2LFjuOOOO9DS0oL3vve9ePLJJ4cssCUiIqKpJ+91VI4H66gQEREVHx5KSERERJMCgwoREREVLAYVIiIiKlgMKkRERFSwGFSIiIioYDGoEBERUcFiUCEiIqKCxaBCREREBYtBhYiIiApW3kvoH49EUV2/35/nlhAREVGmEp/bmRTHL+qgEggEAACNjY15bgkRERGNVSAQQGVl5YjXFPVZP67r4siRIygvL4cQIqev7ff70djYiObmZp4jNI7YzxOD/Twx2M8Th309Mcarn5VSCAQCmDlzJqQceRVKUY+oSCkxa9ascf0eFRUV/D/BBGA/Twz288RgP08c9vXEGI9+Hm0kJYGLaYmIiKhgMagQERFRwWJQGYZpmvh//+//wTTNfDdlUmM/Twz288RgP08c9vXEKIR+LurFtERERDS5cUSFiIiIChaDChERERUsBhUiIiIqWAwqREREVLAYVNL4wQ9+gLlz58Lr9WLlypV49dVX892kotbU1IQzzjgD5eXlqKurwyc+8Qns3bs35ZpwOIy1a9eiuroaZWVl+NSnPoXW1tY8tXhy2LBhA4QQuOmmm5L3sZ9z591338XnPvc5VFdXw+fzYcmSJdi2bVvycaUU7rjjDsyYMQM+nw8XXHAB9u3bl8cWFx/HcXD77bdj3rx58Pl8mD9/Pu66666U82HYz2P3wgsv4JJLLsHMmTMhhMDjjz+e8ngmfdrZ2YnVq1ejoqICVVVVuPbaa9Hb2zs+DVaU4uGHH1Yej0f97//+r3rjjTfUv/zLv6iqqirV2tqa76YVrY985CPq/vvvV7t27VI7duxQH/vYx9Ts2bNVb29v8prrrrtONTY2qk2bNqlt27ap973vfeqss87KY6uL26uvvqrmzp2rli5dqm688cbk/ezn3Ojs7FRz5sxRX/jCF9Qrr7yiDhw4oJ566im1f//+5DUbNmxQlZWV6vHHH1evv/66uvTSS9W8efNUKBTKY8uLy913362qq6vVH/7wB3Xw4EH1q1/9SpWVlanvfe97yWvYz2P3xBNPqNtuu0395je/UQDUY489lvJ4Jn360Y9+VC1btkxt2bJF/eUvf1ELFixQV1111bi0l0FlkDPPPFOtXbs2+bXjOGrmzJmqqakpj62aXNra2hQA9fzzzyullOru7laGYahf/epXyWv27NmjAKiXX345X80sWoFAQJ100knq6aefVuedd14yqLCfc+drX/uaOuecc4Z93HVd1dDQoP77v/87eV93d7cyTVM99NBDE9HESeHiiy9W11xzTcp9n/zkJ9Xq1auVUuznXBgcVDLp0927dysAauvWrclr/vSnPykhhHr33Xdz3kZO/QwQiUSwfft2XHDBBcn7pJS44IIL8PLLL+exZZNLT08PAGD69OkAgO3btyMajab0+8KFCzF79mz2exbWrl2Liy++OKU/AfZzLv3ud7/DihUrcMUVV6Curg7Lly/HT37yk+TjBw8eREtLS0pfV1ZWYuXKlezrMTjrrLOwadMmvPXWWwCA119/HS+++CIuuugiAOzn8ZBJn7788suoqqrCihUrktdccMEFkFLilVdeyXmbivpQwlxrb2+H4zior69Pub++vh5vvvlmnlo1ubiui5tuuglnn302Fi9eDABoaWmBx+NBVVVVyrX19fVoaWnJQyuL18MPP4y//e1v2Lp165DH2M+5c+DAAWzcuBG33HILvv71r2Pr1q1Yt24dPB4P1qxZk+zPdP8tYV9n7tZbb4Xf78fChQuhaRocx8Hdd9+N1atXAwD7eRxk0qctLS2oq6tLeVzXdUyfPn1c+p1BhSbU2rVrsWvXLrz44ov5bsqk09zcjBtvvBFPP/00vF5vvpszqbmuixUrVuBb3/oWAGD58uXYtWsXfvjDH2LNmjV5bt3k8eijj+KXv/wlHnzwQSxatAg7duzATTfdhJkzZ7KfpxBO/QxQU1MDTdOG7IJobW1FQ0NDnlo1edxwww34wx/+gOeeew6zZs1K3t/Q0IBIJILu7u6U69nvY7N9+3a0tbXhtNNOg67r0HUdzz//PO69917ouo76+nr2c47MmDEDp556asp9p5xyCg4fPgwAyf7kf0uOz1e+8hXceuut+MxnPoMlS5bgn//5n3HzzTejqakJAPt5PGTSpw0NDWhra0t53LZtdHZ2jku/M6gM4PF4cPrpp2PTpk3J+1zXxaZNm7Bq1ao8tqy4KaVwww034LHHHsOzzz6LefPmpTx++umnwzCMlH7fu3cvDh8+zH4fg/PPPx87d+7Ejh07krcVK1Zg9erVyb+zn3Pj7LPPHrLF/q233sKcOXMAAPPmzUNDQ0NKX/v9frzyyivs6zHo6+uDlKkfU5qmwXVdAOzn8ZBJn65atQrd3d3Yvn178ppnn30Wruti5cqVuW9UzpfnFrmHH35YmaapHnjgAbV79271r//6r6qqqkq1tLTku2lF64tf/KKqrKxUmzdvVkePHk3e+vr6ktdcd911avbs2erZZ59V27ZtU6tWrVKrVq3KY6snh4G7fpRiP+fKq6++qnRdV3fffbfat2+f+uUvf6lKSkrUL37xi+Q1GzZsUFVVVeq3v/2t+vvf/64uu+wybpsdozVr1qgTTjghuT35N7/5jaqpqVFf/epXk9ewn8cuEAio1157Tb322msKgPrOd76jXnvtNXXo0CGlVGZ9+tGPflQtX75cvfLKK+rFF19UJ510ErcnT6Tvf//7avbs2crj8agzzzxTbdmyJd9NKmoA0t7uv//+5DWhUEhdf/31atq0aaqkpERdfvnl6ujRo/lr9CQxOKiwn3Pn97//vVq8eLEyTVMtXLhQ/fjHP0553HVddfvtt6v6+nplmqY6//zz1d69e/PU2uLk9/vVjTfeqGbPnq28Xq868cQT1W233aYsy0pew34eu+eeey7tf5PXrFmjlMqsTzs6OtRVV12lysrKVEVFhbr66qtVIBAYl/YKpQaU+CMiIiIqIFyjQkRERAWLQYWIiIgKFoMKERERFSwGFSIiIipYDCpERERUsBhUiIiIqGAxqBAREVHBYlAhIiKigsWgQkSTyubNmyGEGHL4IhEVJwYVIiIiKlgMKkRERFSwGFSIKKdc10VTUxPmzZsHn8+HZcuW4de//jWA/mmZP/7xj1i6dCm8Xi/e9773YdeuXSmv8X//939YtGgRTNPE3Llz8e1vfzvlccuy8LWvfQ2NjY0wTRMLFizAT3/605Rrtm/fjhUrVqCkpARnnXUW9u7dO75vnIjGBYMKEeVUU1MTfv7zn+OHP/wh3njjDdx888343Oc+h+effz55zVe+8hV8+9vfxtatW1FbW4tLLrkE0WgUQCxgXHnllfjMZz6DnTt34hvf+AZuv/12PPDAA8nnf/7zn8dDDz2Ee++9F3v27MGPfvQjlJWVpbTjtttuw7e//W1s27YNuq7jmmuumZD3T0Q5Ni5nMhPRlBQOh1VJSYn661//mnL/tddeq6666qrk8fIPP/xw8rGOjg7l8/nUI488opRS6rOf/ay68MILU57/la98RZ166qlKKaX27t2rAKinn346bRsS3+OZZ55J3vfHP/5RAVChUCgn75OIJg5HVIgoZ/bv34++vj5ceOGFKCsrS95+/vOf4+23305et2rVquTfp0+fjve85z3Ys2cPAGDPnj04++yzU1737LPPxr59++A4Dnbs2AFN03DeeeeN2JalS5cm/z5jxgwAQFtb23G/RyKaWHq+G0BEk0dvby8A4I9//CNOOOGElMdM00wJK9ny+XwZXWcYRvLvQggAsfUzRFRcOKJCRDlz6qmnwjRNHD58GAsWLEi5NTY2Jq/bsmVL8u9dXV146623cMoppwAATjnlFLz00kspr/vSSy/h5JNPhqZpWLJkCVzXTVnzQkSTF0dUiChnysvL8e///u+4+eab4bouzjnnHPT09OCll15CRUUF5syZAwC48847UV1djfr6etx2222oqanBJz7xCQDAl7/8ZZxxxhm466678OlPfxovv/wy7rvvPvzP//wPAGDu3LlYs2YNrrnmGtx7771YtmwZDh06hLa2Nlx55ZX5eutENE4YVIgop+666y7U1taiqakJBw4cQFVVFU477TR8/etfT069bNiwATfeeCP27duH9773vfj9738Pj8cDADjttNPw6KOP4o477sBdd92FGTNm4M4778QXvvCF5PfYuHEjvv71r+P6669HR0cHZs+eja9//ev5eLtENM6EUkrluxFENDVs3rwZH/zgB9HV1YWqqqp8N4eIigDXqBAREVHBYlAhIiKigsWpHyIiIipYHFEhIiKigsWgQkRERAWLQYWIiIgKFoMKERERFSwGFSIiIipYDCpERERUsBhUiIiIqGAxqBAREVHB+v+uSwtw2Vh4XwAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 640x480 with 1 Axes>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "import pandas as pd\n",
        "import seaborn as sns\n",
        "\n",
        "model = operation.result()\n",
        "\n",
        "snapshots = pd.DataFrame(model.tuning_task.snapshots)\n",
        "\n",
        "sns.lineplot(data=snapshots, x = 'epoch', y='mean_loss')\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rkoQTXb1vSBC"
      },
      "source": [
        "## Evaluate your model\n",
        "\n",
        "You can use the `genai.generate_text` method and specify the name of your model to test your model performance."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "zO0YcuSyxydZ"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'6'"
            ]
          },
          "execution_count": 18,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "completion = genai.generate_text(model=f'tunedModels/{name}',\n",
        "                                prompt='5')\n",
        "completion.result"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "YSNB2zjTx5SZ"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'-8'"
            ]
          },
          "execution_count": 19,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "completion = genai.generate_text(model=f'tunedModels/{name}',\n",
        "                                prompt='-9')\n",
        "completion.result"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Y2YVO-m0Ut9H"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'four'"
            ]
          },
          "execution_count": 20,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "completion = genai.generate_text(model=f'tunedModels/{name}',\n",
        "                                prompt='four')\n",
        "completion.result"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HpIA1IFevQQR"
      },
      "source": [
        "As you can see, the last prompt didn't produce the ideal result, `five`. To produce better results you can try a few different things such as adjusting the temperature closer to zero to get more consistent results, adding more quality examples to your dataset that the model can learn from or adding a prompt or preamble to the examples.\n",
        "\n",
        "See the [tuning guide](https://ai.google.dev/gemini-api/docs/model-tuning) for more guidance on improving performance."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nmuQCbTYwIOx"
      },
      "source": [
        "## Update the description\n",
        "\n",
        "You can update the description of your tuned model any time using the `genai.update_tuned_model` method."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "9gAVuXT_wG3x"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "TunedModel(name='', source_model=None, base_model=None, display_name='', description='This is my model.', temperature=None, top_p=None, top_k=None, state=<State.STATE_UNSPECIFIED: 0>, create_time=None, update_time=None, tuning_task=None)"
            ]
          },
          "execution_count": 21,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "genai.update_tuned_model(f'tunedModels/{name}', {\"description\":\"This is my model.\"})"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "d-c3YerBxVYs"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "TunedModel(name='tunedModels/generate-num-4668',\n",
            "           source_model=None,\n",
            "           base_model='models/text-bison-001',\n",
            "           display_name='',\n",
            "           description='This is my model.',\n",
            "           temperature=0.7,\n",
            "           top_p=0.95,\n",
            "           top_k=40,\n",
            "           state=<State.ACTIVE: 2>,\n",
            "           create_time=datetime.datetime(2023, 9, 19, 19, 3, 38, 22249, tzinfo=<UTC>),\n",
            "           update_time=datetime.datetime(2023, 9, 19, 19, 11, 48, 101024, tzinfo=<UTC>),\n",
            "           tuning_task=TuningTask(start_time=datetime.datetime(2023, 9, 19, 19, 3, 38, 562798, tzinfo=<UTC>),\n",
            "                                  complete_time=datetime.datetime(2023, 9, 19, 19, 11, 48, 101024, tzinfo=<UTC>),\n",
            "                                  snapshots=[{'compute_time': datetime.datetime(2023, 9, 19, 19, 3, 41, 221503, tzinfo=<UTC>),\n",
            "                                              'epoch': 0,\n",
            "                                              'mean_loss': 7.2774773,\n",
            "                                              'step': 1},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 3, 42, 611142, tzinfo=<UTC>),\n",
            "                                              'epoch': 0,\n",
            "                                              'mean_loss': 6.178241,\n",
            "                                              'step': 2},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 3, 43, 886844, tzinfo=<UTC>),\n",
            "                                              'epoch': 0,\n",
            "                                              'mean_loss': 5.505934,\n",
            "                                              'step': 3},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 3, 45, 213316, tzinfo=<UTC>),\n",
            "                                              'epoch': 1,\n",
            "                                              'mean_loss': 7.9365344,\n",
            "                                              'step': 4},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 3, 46, 719674, tzinfo=<UTC>),\n",
            "                                              'epoch': 1,\n",
            "                                              'mean_loss': 7.656596,\n",
            "                                              'step': 5},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 3, 48, 51559, tzinfo=<UTC>),\n",
            "                                              'epoch': 1,\n",
            "                                              'mean_loss': 7.3750257,\n",
            "                                              'step': 6},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 3, 49, 419247, tzinfo=<UTC>),\n",
            "                                              'epoch': 1,\n",
            "                                              'mean_loss': 4.579882,\n",
            "                                              'step': 7},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 3, 50, 902477, tzinfo=<UTC>),\n",
            "                                              'epoch': 2,\n",
            "                                              'mean_loss': 6.776862,\n",
            "                                              'step': 8},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 3, 52, 213448, tzinfo=<UTC>),\n",
            "                                              'epoch': 2,\n",
            "                                              'mean_loss': 6.3564157,\n",
            "                                              'step': 9},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 3, 53, 679693, tzinfo=<UTC>),\n",
            "                                              'epoch': 2,\n",
            "                                              'mean_loss': 8.558726,\n",
            "                                              'step': 10},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 3, 55, 2348, tzinfo=<UTC>),\n",
            "                                              'epoch': 2,\n",
            "                                              'mean_loss': 4.783774,\n",
            "                                              'step': 11},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 3, 56, 322485, tzinfo=<UTC>),\n",
            "                                              'epoch': 3,\n",
            "                                              'mean_loss': 7.0234137,\n",
            "                                              'step': 12},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 3, 58, 145081, tzinfo=<UTC>),\n",
            "                                              'epoch': 3,\n",
            "                                              'mean_loss': 7.317513,\n",
            "                                              'step': 13},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 3, 59, 399317, tzinfo=<UTC>),\n",
            "                                              'epoch': 3,\n",
            "                                              'mean_loss': 5.85363,\n",
            "                                              'step': 14},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 0, 646995, tzinfo=<UTC>),\n",
            "                                              'epoch': 4,\n",
            "                                              'mean_loss': 4.21408,\n",
            "                                              'step': 15},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 1, 899798, tzinfo=<UTC>),\n",
            "                                              'epoch': 4,\n",
            "                                              'mean_loss': 6.6232214,\n",
            "                                              'step': 16},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 3, 167955, tzinfo=<UTC>),\n",
            "                                              'epoch': 4,\n",
            "                                              'mean_loss': 5.61497,\n",
            "                                              'step': 17},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 4, 407849, tzinfo=<UTC>),\n",
            "                                              'epoch': 4,\n",
            "                                              'mean_loss': 6.821261,\n",
            "                                              'step': 18},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 5, 649503, tzinfo=<UTC>),\n",
            "                                              'epoch': 5,\n",
            "                                              'mean_loss': 3.8338904,\n",
            "                                              'step': 19},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 7, 80497, tzinfo=<UTC>),\n",
            "                                              'epoch': 5,\n",
            "                                              'mean_loss': 5.0643735,\n",
            "                                              'step': 20},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 8, 401424, tzinfo=<UTC>),\n",
            "                                              'epoch': 5,\n",
            "                                              'mean_loss': 6.976447,\n",
            "                                              'step': 21},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 9, 688226, tzinfo=<UTC>),\n",
            "                                              'epoch': 5,\n",
            "                                              'mean_loss': 5.045044,\n",
            "                                              'step': 22},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 10, 942147, tzinfo=<UTC>),\n",
            "                                              'epoch': 6,\n",
            "                                              'mean_loss': 5.1944356,\n",
            "                                              'step': 23},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 12, 169501, tzinfo=<UTC>),\n",
            "                                              'epoch': 6,\n",
            "                                              'mean_loss': 5.342552,\n",
            "                                              'step': 24},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 13, 532023, tzinfo=<UTC>),\n",
            "                                              'epoch': 6,\n",
            "                                              'mean_loss': 7.360283,\n",
            "                                              'step': 25},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 14, 773265, tzinfo=<UTC>),\n",
            "                                              'epoch': 6,\n",
            "                                              'mean_loss': 2.874686,\n",
            "                                              'step': 26},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 16, 68826, tzinfo=<UTC>),\n",
            "                                              'epoch': 7,\n",
            "                                              'mean_loss': 5.0835795,\n",
            "                                              'step': 27},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 17, 328292, tzinfo=<UTC>),\n",
            "                                              'epoch': 7,\n",
            "                                              'mean_loss': 4.059507,\n",
            "                                              'step': 28},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 18, 683769, tzinfo=<UTC>),\n",
            "                                              'epoch': 7,\n",
            "                                              'mean_loss': 4.668791,\n",
            "                                              'step': 29},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 19, 917365, tzinfo=<UTC>),\n",
            "                                              'epoch': 8,\n",
            "                                              'mean_loss': 3.2776065,\n",
            "                                              'step': 30},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 21, 175338, tzinfo=<UTC>),\n",
            "                                              'epoch': 8,\n",
            "                                              'mean_loss': 4.1344976,\n",
            "                                              'step': 31},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 22, 510908, tzinfo=<UTC>),\n",
            "                                              'epoch': 8,\n",
            "                                              'mean_loss': 4.47365,\n",
            "                                              'step': 32},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 23, 972490, tzinfo=<UTC>),\n",
            "                                              'epoch': 8,\n",
            "                                              'mean_loss': 2.8087254,\n",
            "                                              'step': 33},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 25, 341109, tzinfo=<UTC>),\n",
            "                                              'epoch': 9,\n",
            "                                              'mean_loss': 3.581566,\n",
            "                                              'step': 34},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 26, 594799, tzinfo=<UTC>),\n",
            "                                              'epoch': 9,\n",
            "                                              'mean_loss': 3.3534799,\n",
            "                                              'step': 35},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 27, 857511, tzinfo=<UTC>),\n",
            "                                              'epoch': 9,\n",
            "                                              'mean_loss': 2.5248497,\n",
            "                                              'step': 36},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 29, 100872, tzinfo=<UTC>),\n",
            "                                              'epoch': 9,\n",
            "                                              'mean_loss': 1.8420736,\n",
            "                                              'step': 37},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 30, 356383, tzinfo=<UTC>),\n",
            "                                              'epoch': 10,\n",
            "                                              'mean_loss': 3.4610085,\n",
            "                                              'step': 38},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 31, 819918, tzinfo=<UTC>),\n",
            "                                              'epoch': 10,\n",
            "                                              'mean_loss': 3.2506752,\n",
            "                                              'step': 39},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 33, 77814, tzinfo=<UTC>),\n",
            "                                              'epoch': 10,\n",
            "                                              'mean_loss': 2.4844272,\n",
            "                                              'step': 40},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 34, 314311, tzinfo=<UTC>),\n",
            "                                              'epoch': 10,\n",
            "                                              'mean_loss': 2.3858242,\n",
            "                                              'step': 41},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 35, 572181, tzinfo=<UTC>),\n",
            "                                              'epoch': 11,\n",
            "                                              'mean_loss': 1.1961311,\n",
            "                                              'step': 42},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 36, 845239, tzinfo=<UTC>),\n",
            "                                              'epoch': 11,\n",
            "                                              'mean_loss': 3.5777583,\n",
            "                                              'step': 43},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 38, 120182, tzinfo=<UTC>),\n",
            "                                              'epoch': 11,\n",
            "                                              'mean_loss': 1.3613169,\n",
            "                                              'step': 44},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 39, 611773, tzinfo=<UTC>),\n",
            "                                              'epoch': 12,\n",
            "                                              'mean_loss': 1.7414228,\n",
            "                                              'step': 45},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 41, 835960, tzinfo=<UTC>),\n",
            "                                              'epoch': 12,\n",
            "                                              'mean_loss': 1.3301177,\n",
            "                                              'step': 46},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 43, 118015, tzinfo=<UTC>),\n",
            "                                              'epoch': 12,\n",
            "                                              'mean_loss': 1.3805578,\n",
            "                                              'step': 47},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 44, 383045, tzinfo=<UTC>),\n",
            "                                              'epoch': 12,\n",
            "                                              'mean_loss': 2.3191347,\n",
            "                                              'step': 48},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 45, 617675, tzinfo=<UTC>),\n",
            "                                              'epoch': 13,\n",
            "                                              'mean_loss': 1.7018254,\n",
            "                                              'step': 49},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 46, 856463, tzinfo=<UTC>),\n",
            "                                              'epoch': 13,\n",
            "                                              'mean_loss': 1.5530272,\n",
            "                                              'step': 50},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 48, 159606, tzinfo=<UTC>),\n",
            "                                              'epoch': 13,\n",
            "                                              'mean_loss': 2.1536818,\n",
            "                                              'step': 51},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 49, 388434, tzinfo=<UTC>),\n",
            "                                              'epoch': 13,\n",
            "                                              'mean_loss': 0.87225634,\n",
            "                                              'step': 52},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 50, 649576, tzinfo=<UTC>),\n",
            "                                              'epoch': 14,\n",
            "                                              'mean_loss': 1.6638466,\n",
            "                                              'step': 53},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 52, 113467, tzinfo=<UTC>),\n",
            "                                              'epoch': 14,\n",
            "                                              'mean_loss': 1.4672767,\n",
            "                                              'step': 54},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 53, 491995, tzinfo=<UTC>),\n",
            "                                              'epoch': 14,\n",
            "                                              'mean_loss': 0.66232294,\n",
            "                                              'step': 55},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 54, 849227, tzinfo=<UTC>),\n",
            "                                              'epoch': 14,\n",
            "                                              'mean_loss': 1.2151186,\n",
            "                                              'step': 56},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 56, 117613, tzinfo=<UTC>),\n",
            "                                              'epoch': 15,\n",
            "                                              'mean_loss': 0.75382125,\n",
            "                                              'step': 57},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 58, 244537, tzinfo=<UTC>),\n",
            "                                              'epoch': 15,\n",
            "                                              'mean_loss': 0.909588,\n",
            "                                              'step': 58},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 4, 59, 495142, tzinfo=<UTC>),\n",
            "                                              'epoch': 15,\n",
            "                                              'mean_loss': 0.85212016,\n",
            "                                              'step': 59},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 0, 748073, tzinfo=<UTC>),\n",
            "                                              'epoch': 16,\n",
            "                                              'mean_loss': 1.0999682,\n",
            "                                              'step': 60},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 2, 9621, tzinfo=<UTC>),\n",
            "                                              'epoch': 16,\n",
            "                                              'mean_loss': 0.49189907,\n",
            "                                              'step': 61},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 3, 289800, tzinfo=<UTC>),\n",
            "                                              'epoch': 16,\n",
            "                                              'mean_loss': 1.2313881,\n",
            "                                              'step': 62},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 4, 542260, tzinfo=<UTC>),\n",
            "                                              'epoch': 16,\n",
            "                                              'mean_loss': 0.4186042,\n",
            "                                              'step': 63},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 5, 789722, tzinfo=<UTC>),\n",
            "                                              'epoch': 17,\n",
            "                                              'mean_loss': 0.5968985,\n",
            "                                              'step': 64},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 7, 21547, tzinfo=<UTC>),\n",
            "                                              'epoch': 17,\n",
            "                                              'mean_loss': 0.32776576,\n",
            "                                              'step': 65},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 8, 253903, tzinfo=<UTC>),\n",
            "                                              'epoch': 17,\n",
            "                                              'mean_loss': 0.085846476,\n",
            "                                              'step': 66},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 9, 503217, tzinfo=<UTC>),\n",
            "                                              'epoch': 17,\n",
            "                                              'mean_loss': 0.87150824,\n",
            "                                              'step': 67},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 10, 755627, tzinfo=<UTC>),\n",
            "                                              'epoch': 18,\n",
            "                                              'mean_loss': 0.50882834,\n",
            "                                              'step': 68},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 11, 981698, tzinfo=<UTC>),\n",
            "                                              'epoch': 18,\n",
            "                                              'mean_loss': 0.05643571,\n",
            "                                              'step': 69},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 13, 238454, tzinfo=<UTC>),\n",
            "                                              'epoch': 18,\n",
            "                                              'mean_loss': 0.11667071,\n",
            "                                              'step': 70},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 14, 474345, tzinfo=<UTC>),\n",
            "                                              'epoch': 18,\n",
            "                                              'mean_loss': 0.05200408,\n",
            "                                              'step': 71},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 15, 692710, tzinfo=<UTC>),\n",
            "                                              'epoch': 19,\n",
            "                                              'mean_loss': 0.21968448,\n",
            "                                              'step': 72},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 16, 930777, tzinfo=<UTC>),\n",
            "                                              'epoch': 19,\n",
            "                                              'mean_loss': 0.071391255,\n",
            "                                              'step': 73},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 18, 180590, tzinfo=<UTC>),\n",
            "                                              'epoch': 19,\n",
            "                                              'mean_loss': 0.39031163,\n",
            "                                              'step': 74},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 19, 433064, tzinfo=<UTC>),\n",
            "                                              'epoch': 20,\n",
            "                                              'mean_loss': 0.05084487,\n",
            "                                              'step': 75},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 20, 677200, tzinfo=<UTC>),\n",
            "                                              'epoch': 20,\n",
            "                                              'mean_loss': 0.04713744,\n",
            "                                              'step': 76},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 21, 901118, tzinfo=<UTC>),\n",
            "                                              'epoch': 20,\n",
            "                                              'mean_loss': 0.196708,\n",
            "                                              'step': 77},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 23, 166260, tzinfo=<UTC>),\n",
            "                                              'epoch': 20,\n",
            "                                              'mean_loss': 0.15159458,\n",
            "                                              'step': 78},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 24, 400680, tzinfo=<UTC>),\n",
            "                                              'epoch': 21,\n",
            "                                              'mean_loss': 0.0280451,\n",
            "                                              'step': 79},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 25, 644378, tzinfo=<UTC>),\n",
            "                                              'epoch': 21,\n",
            "                                              'mean_loss': 0.06759574,\n",
            "                                              'step': 80},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 27, 195128, tzinfo=<UTC>),\n",
            "                                              'epoch': 21,\n",
            "                                              'mean_loss': 0.03170073,\n",
            "                                              'step': 81},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 28, 546850, tzinfo=<UTC>),\n",
            "                                              'epoch': 21,\n",
            "                                              'mean_loss': 0.15327619,\n",
            "                                              'step': 82},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 29, 953511, tzinfo=<UTC>),\n",
            "                                              'epoch': 22,\n",
            "                                              'mean_loss': 0.14349619,\n",
            "                                              'step': 83},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 31, 334082, tzinfo=<UTC>),\n",
            "                                              'epoch': 22,\n",
            "                                              'mean_loss': 0.02684513,\n",
            "                                              'step': 84},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 32, 832994, tzinfo=<UTC>),\n",
            "                                              'epoch': 22,\n",
            "                                              'mean_loss': 0.019874452,\n",
            "                                              'step': 85},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 34, 88577, tzinfo=<UTC>),\n",
            "                                              'epoch': 22,\n",
            "                                              'mean_loss': 0.041133285,\n",
            "                                              'step': 86},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 35, 346650, tzinfo=<UTC>),\n",
            "                                              'epoch': 23,\n",
            "                                              'mean_loss': 0.06348712,\n",
            "                                              'step': 87},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 36, 585951, tzinfo=<UTC>),\n",
            "                                              'epoch': 23,\n",
            "                                              'mean_loss': 0.025213383,\n",
            "                                              'step': 88},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 37, 818161, tzinfo=<UTC>),\n",
            "                                              'epoch': 23,\n",
            "                                              'mean_loss': 0.018140253,\n",
            "                                              'step': 89},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 39, 72239, tzinfo=<UTC>),\n",
            "                                              'epoch': 24,\n",
            "                                              'mean_loss': 0.023763947,\n",
            "                                              'step': 90},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 40, 278252, tzinfo=<UTC>),\n",
            "                                              'epoch': 24,\n",
            "                                              'mean_loss': 0.008751405,\n",
            "                                              'step': 91},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 42, 747639, tzinfo=<UTC>),\n",
            "                                              'epoch': 24,\n",
            "                                              'mean_loss': 0.0082112085,\n",
            "                                              'step': 92},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 44, 8609, tzinfo=<UTC>),\n",
            "                                              'epoch': 24,\n",
            "                                              'mean_loss': 0.037568945,\n",
            "                                              'step': 93},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 45, 264108, tzinfo=<UTC>),\n",
            "                                              'epoch': 25,\n",
            "                                              'mean_loss': 0.027123686,\n",
            "                                              'step': 94},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 46, 526159, tzinfo=<UTC>),\n",
            "                                              'epoch': 25,\n",
            "                                              'mean_loss': 0.0142503055,\n",
            "                                              'step': 95},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 47, 768520, tzinfo=<UTC>),\n",
            "                                              'epoch': 25,\n",
            "                                              'mean_loss': 0.027518341,\n",
            "                                              'step': 96},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 49, 89441, tzinfo=<UTC>),\n",
            "                                              'epoch': 25,\n",
            "                                              'mean_loss': 0.013976067,\n",
            "                                              'step': 97},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 50, 342449, tzinfo=<UTC>),\n",
            "                                              'epoch': 26,\n",
            "                                              'mean_loss': 0.0036393465,\n",
            "                                              'step': 98},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 51, 613018, tzinfo=<UTC>),\n",
            "                                              'epoch': 26,\n",
            "                                              'mean_loss': 0.0058721625,\n",
            "                                              'step': 99},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 52, 847043, tzinfo=<UTC>),\n",
            "                                              'epoch': 26,\n",
            "                                              'mean_loss': 0.0008192812,\n",
            "                                              'step': 100},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 54, 81155, tzinfo=<UTC>),\n",
            "                                              'epoch': 26,\n",
            "                                              'mean_loss': 0.025449298,\n",
            "                                              'step': 101},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 55, 304198, tzinfo=<UTC>),\n",
            "                                              'epoch': 27,\n",
            "                                              'mean_loss': 0.0066863927,\n",
            "                                              'step': 102},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 56, 557883, tzinfo=<UTC>),\n",
            "                                              'epoch': 27,\n",
            "                                              'mean_loss': 0.002721126,\n",
            "                                              'step': 103},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 57, 787163, tzinfo=<UTC>),\n",
            "                                              'epoch': 27,\n",
            "                                              'mean_loss': 0.015793594,\n",
            "                                              'step': 104},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 5, 59, 81532, tzinfo=<UTC>),\n",
            "                                              'epoch': 28,\n",
            "                                              'mean_loss': 0.005504051,\n",
            "                                              'step': 105},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 0, 328216, tzinfo=<UTC>),\n",
            "                                              'epoch': 28,\n",
            "                                              'mean_loss': 0.003016818,\n",
            "                                              'step': 106},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 1, 561541, tzinfo=<UTC>),\n",
            "                                              'epoch': 28,\n",
            "                                              'mean_loss': 0.014285667,\n",
            "                                              'step': 107},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 2, 782923, tzinfo=<UTC>),\n",
            "                                              'epoch': 28,\n",
            "                                              'mean_loss': 0.004257772,\n",
            "                                              'step': 108},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 4, 10208, tzinfo=<UTC>),\n",
            "                                              'epoch': 29,\n",
            "                                              'mean_loss': 0.010257841,\n",
            "                                              'step': 109},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 5, 268123, tzinfo=<UTC>),\n",
            "                                              'epoch': 29,\n",
            "                                              'mean_loss': 0.0075931884,\n",
            "                                              'step': 110},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 6, 485682, tzinfo=<UTC>),\n",
            "                                              'epoch': 29,\n",
            "                                              'mean_loss': 0.024589492,\n",
            "                                              'step': 111},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 7, 731568, tzinfo=<UTC>),\n",
            "                                              'epoch': 29,\n",
            "                                              'mean_loss': 0.0012908785,\n",
            "                                              'step': 112},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 8, 978691, tzinfo=<UTC>),\n",
            "                                              'epoch': 30,\n",
            "                                              'mean_loss': 0.011656972,\n",
            "                                              'step': 113},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 10, 254824, tzinfo=<UTC>),\n",
            "                                              'epoch': 30,\n",
            "                                              'mean_loss': 0.006077944,\n",
            "                                              'step': 114},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 11, 489143, tzinfo=<UTC>),\n",
            "                                              'epoch': 30,\n",
            "                                              'mean_loss': 0.004817399,\n",
            "                                              'step': 115},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 12, 772297, tzinfo=<UTC>),\n",
            "                                              'epoch': 30,\n",
            "                                              'mean_loss': 0.007809804,\n",
            "                                              'step': 116},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 14, 19305, tzinfo=<UTC>),\n",
            "                                              'epoch': 31,\n",
            "                                              'mean_loss': 0.006533157,\n",
            "                                              'step': 117},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 15, 257150, tzinfo=<UTC>),\n",
            "                                              'epoch': 31,\n",
            "                                              'mean_loss': 0.005006207,\n",
            "                                              'step': 118},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 16, 536830, tzinfo=<UTC>),\n",
            "                                              'epoch': 31,\n",
            "                                              'mean_loss': 0.0004677312,\n",
            "                                              'step': 119},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 17, 790581, tzinfo=<UTC>),\n",
            "                                              'epoch': 32,\n",
            "                                              'mean_loss': 0.007230793,\n",
            "                                              'step': 120},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 19, 41049, tzinfo=<UTC>),\n",
            "                                              'epoch': 32,\n",
            "                                              'mean_loss': 0.0020942457,\n",
            "                                              'step': 121},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 20, 306408, tzinfo=<UTC>),\n",
            "                                              'epoch': 32,\n",
            "                                              'mean_loss': 0.006205416,\n",
            "                                              'step': 122},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 21, 545345, tzinfo=<UTC>),\n",
            "                                              'epoch': 32,\n",
            "                                              'mean_loss': 0.0063284263,\n",
            "                                              'step': 123},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 22, 759006, tzinfo=<UTC>),\n",
            "                                              'epoch': 33,\n",
            "                                              'mean_loss': 0.0063140467,\n",
            "                                              'step': 124},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 23, 979661, tzinfo=<UTC>),\n",
            "                                              'epoch': 33,\n",
            "                                              'mean_loss': 0.00040962745,\n",
            "                                              'step': 125},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 25, 215222, tzinfo=<UTC>),\n",
            "                                              'epoch': 33,\n",
            "                                              'mean_loss': 0.0020570084,\n",
            "                                              'step': 126},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 26, 457055, tzinfo=<UTC>),\n",
            "                                              'epoch': 33,\n",
            "                                              'mean_loss': 0.011655594,\n",
            "                                              'step': 127},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 27, 705186, tzinfo=<UTC>),\n",
            "                                              'epoch': 34,\n",
            "                                              'mean_loss': 0.0038693137,\n",
            "                                              'step': 128},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 29, 602189, tzinfo=<UTC>),\n",
            "                                              'epoch': 34,\n",
            "                                              'mean_loss': 0.0092602,\n",
            "                                              'step': 129},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 30, 874690, tzinfo=<UTC>),\n",
            "                                              'epoch': 34,\n",
            "                                              'mean_loss': 0.0047834637,\n",
            "                                              'step': 130},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 32, 120016, tzinfo=<UTC>),\n",
            "                                              'epoch': 34,\n",
            "                                              'mean_loss': 0.00579008,\n",
            "                                              'step': 131},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 33, 432962, tzinfo=<UTC>),\n",
            "                                              'epoch': 35,\n",
            "                                              'mean_loss': 0.004663332,\n",
            "                                              'step': 132},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 34, 709379, tzinfo=<UTC>),\n",
            "                                              'epoch': 35,\n",
            "                                              'mean_loss': 0.004609281,\n",
            "                                              'step': 133},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 35, 948660, tzinfo=<UTC>),\n",
            "                                              'epoch': 35,\n",
            "                                              'mean_loss': 0.0044703777,\n",
            "                                              'step': 134},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 37, 173785, tzinfo=<UTC>),\n",
            "                                              'epoch': 36,\n",
            "                                              'mean_loss': 0.0016537609,\n",
            "                                              'step': 135},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 38, 412757, tzinfo=<UTC>),\n",
            "                                              'epoch': 36,\n",
            "                                              'mean_loss': 0.005695714,\n",
            "                                              'step': 136},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 39, 648122, tzinfo=<UTC>),\n",
            "                                              'epoch': 36,\n",
            "                                              'mean_loss': 0.004103207,\n",
            "                                              'step': 137},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 40, 885574, tzinfo=<UTC>),\n",
            "                                              'epoch': 36,\n",
            "                                              'mean_loss': 0.0018846963,\n",
            "                                              'step': 138},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 42, 993050, tzinfo=<UTC>),\n",
            "                                              'epoch': 37,\n",
            "                                              'mean_loss': 0.0023978357,\n",
            "                                              'step': 139},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 44, 304394, tzinfo=<UTC>),\n",
            "                                              'epoch': 37,\n",
            "                                              'mean_loss': 0.0033069355,\n",
            "                                              'step': 140},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 45, 565453, tzinfo=<UTC>),\n",
            "                                              'epoch': 37,\n",
            "                                              'mean_loss': 6.576523e-05,\n",
            "                                              'step': 141},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 47, 38618, tzinfo=<UTC>),\n",
            "                                              'epoch': 37,\n",
            "                                              'mean_loss': 0.0043391315,\n",
            "                                              'step': 142},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 48, 351795, tzinfo=<UTC>),\n",
            "                                              'epoch': 38,\n",
            "                                              'mean_loss': 0.0046651517,\n",
            "                                              'step': 143},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 49, 602102, tzinfo=<UTC>),\n",
            "                                              'epoch': 38,\n",
            "                                              'mean_loss': 0.0052327495,\n",
            "                                              'step': 144},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 50, 976143, tzinfo=<UTC>),\n",
            "                                              'epoch': 38,\n",
            "                                              'mean_loss': 0.013687609,\n",
            "                                              'step': 145},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 52, 227875, tzinfo=<UTC>),\n",
            "                                              'epoch': 38,\n",
            "                                              'mean_loss': 9.1750175e-05,\n",
            "                                              'step': 146},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 53, 495933, tzinfo=<UTC>),\n",
            "                                              'epoch': 39,\n",
            "                                              'mean_loss': 0.0025392435,\n",
            "                                              'step': 147},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 54, 736253, tzinfo=<UTC>),\n",
            "                                              'epoch': 39,\n",
            "                                              'mean_loss': 0.0011976548,\n",
            "                                              'step': 148},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 56, 2776, tzinfo=<UTC>),\n",
            "                                              'epoch': 39,\n",
            "                                              'mean_loss': 0.0025211202,\n",
            "                                              'step': 149},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 57, 245352, tzinfo=<UTC>),\n",
            "                                              'epoch': 40,\n",
            "                                              'mean_loss': 0.0018665651,\n",
            "                                              'step': 150},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 58, 506403, tzinfo=<UTC>),\n",
            "                                              'epoch': 40,\n",
            "                                              'mean_loss': 0.0033759787,\n",
            "                                              'step': 151},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 6, 59, 775888, tzinfo=<UTC>),\n",
            "                                              'epoch': 40,\n",
            "                                              'mean_loss': 0.00062831433,\n",
            "                                              'step': 152},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 1, 18295, tzinfo=<UTC>),\n",
            "                                              'epoch': 40,\n",
            "                                              'mean_loss': 0.0020364965,\n",
            "                                              'step': 153},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 2, 268012, tzinfo=<UTC>),\n",
            "                                              'epoch': 41,\n",
            "                                              'mean_loss': 0.0006586923,\n",
            "                                              'step': 154},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 3, 484102, tzinfo=<UTC>),\n",
            "                                              'epoch': 41,\n",
            "                                              'mean_loss': 0.0031121888,\n",
            "                                              'step': 155},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 4, 776770, tzinfo=<UTC>),\n",
            "                                              'epoch': 41,\n",
            "                                              'mean_loss': 0.0027823262,\n",
            "                                              'step': 156},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 6, 19324, tzinfo=<UTC>),\n",
            "                                              'epoch': 41,\n",
            "                                              'mean_loss': 0.0006065498,\n",
            "                                              'step': 157},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 7, 268435, tzinfo=<UTC>),\n",
            "                                              'epoch': 42,\n",
            "                                              'mean_loss': 0.0023070213,\n",
            "                                              'step': 158},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 8, 512407, tzinfo=<UTC>),\n",
            "                                              'epoch': 42,\n",
            "                                              'mean_loss': 0.0016650247,\n",
            "                                              'step': 159},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 9, 745756, tzinfo=<UTC>),\n",
            "                                              'epoch': 42,\n",
            "                                              'mean_loss': 7.3560514e-06,\n",
            "                                              'step': 160},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 10, 987670, tzinfo=<UTC>),\n",
            "                                              'epoch': 42,\n",
            "                                              'mean_loss': 0.012237127,\n",
            "                                              'step': 161},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 12, 219737, tzinfo=<UTC>),\n",
            "                                              'epoch': 43,\n",
            "                                              'mean_loss': 0.0013661574,\n",
            "                                              'step': 162},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 13, 484756, tzinfo=<UTC>),\n",
            "                                              'epoch': 43,\n",
            "                                              'mean_loss': 0.0064119953,\n",
            "                                              'step': 163},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 14, 730503, tzinfo=<UTC>),\n",
            "                                              'epoch': 43,\n",
            "                                              'mean_loss': 0.0035314618,\n",
            "                                              'step': 164},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 15, 947765, tzinfo=<UTC>),\n",
            "                                              'epoch': 44,\n",
            "                                              'mean_loss': 3.298011e-05,\n",
            "                                              'step': 165},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 17, 190398, tzinfo=<UTC>),\n",
            "                                              'epoch': 44,\n",
            "                                              'mean_loss': 0.00053371524,\n",
            "                                              'step': 166},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 18, 433536, tzinfo=<UTC>),\n",
            "                                              'epoch': 44,\n",
            "                                              'mean_loss': 0.00032033096,\n",
            "                                              'step': 167},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 19, 714514, tzinfo=<UTC>),\n",
            "                                              'epoch': 44,\n",
            "                                              'mean_loss': 0.0019390727,\n",
            "                                              'step': 168},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 20, 935080, tzinfo=<UTC>),\n",
            "                                              'epoch': 45,\n",
            "                                              'mean_loss': 2.4262117e-05,\n",
            "                                              'step': 169},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 22, 167896, tzinfo=<UTC>),\n",
            "                                              'epoch': 45,\n",
            "                                              'mean_loss': 0.00012971938,\n",
            "                                              'step': 170},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 23, 410928, tzinfo=<UTC>),\n",
            "                                              'epoch': 45,\n",
            "                                              'mean_loss': 0.0009971312,\n",
            "                                              'step': 171},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 24, 649509, tzinfo=<UTC>),\n",
            "                                              'epoch': 45,\n",
            "                                              'mean_loss': 0.0011956232,\n",
            "                                              'step': 172},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 25, 860363, tzinfo=<UTC>),\n",
            "                                              'epoch': 46,\n",
            "                                              'mean_loss': 8.339065e-05,\n",
            "                                              'step': 173},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 27, 129342, tzinfo=<UTC>),\n",
            "                                              'epoch': 46,\n",
            "                                              'mean_loss': 0.0013557925,\n",
            "                                              'step': 174},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 28, 356351, tzinfo=<UTC>),\n",
            "                                              'epoch': 46,\n",
            "                                              'mean_loss': 0.00077356555,\n",
            "                                              'step': 175},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 29, 568726, tzinfo=<UTC>),\n",
            "                                              'epoch': 46,\n",
            "                                              'mean_loss': 0.0017021206,\n",
            "                                              'step': 176},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 30, 835703, tzinfo=<UTC>),\n",
            "                                              'epoch': 47,\n",
            "                                              'mean_loss': 0.0010255948,\n",
            "                                              'step': 177},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 32, 82326, tzinfo=<UTC>),\n",
            "                                              'epoch': 47,\n",
            "                                              'mean_loss': 0.003208516,\n",
            "                                              'step': 178},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 33, 301035, tzinfo=<UTC>),\n",
            "                                              'epoch': 47,\n",
            "                                              'mean_loss': 0.0021095886,\n",
            "                                              'step': 179},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 34, 589684, tzinfo=<UTC>),\n",
            "                                              'epoch': 48,\n",
            "                                              'mean_loss': 0.0018085723,\n",
            "                                              'step': 180},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 35, 811121, tzinfo=<UTC>),\n",
            "                                              'epoch': 48,\n",
            "                                              'mean_loss': 0.003906385,\n",
            "                                              'step': 181},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 37, 51086, tzinfo=<UTC>),\n",
            "                                              'epoch': 48,\n",
            "                                              'mean_loss': 0.0081788115,\n",
            "                                              'step': 182},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 38, 296512, tzinfo=<UTC>),\n",
            "                                              'epoch': 48,\n",
            "                                              'mean_loss': 0.0021692181,\n",
            "                                              'step': 183},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 39, 576450, tzinfo=<UTC>),\n",
            "                                              'epoch': 49,\n",
            "                                              'mean_loss': 0.0007408549,\n",
            "                                              'step': 184},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 40, 818968, tzinfo=<UTC>),\n",
            "                                              'epoch': 49,\n",
            "                                              'mean_loss': 0.0005012541,\n",
            "                                              'step': 185},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 42, 115736, tzinfo=<UTC>),\n",
            "                                              'epoch': 49,\n",
            "                                              'mean_loss': 0.0015736766,\n",
            "                                              'step': 186},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 44, 177425, tzinfo=<UTC>),\n",
            "                                              'epoch': 49,\n",
            "                                              'mean_loss': 0.0010295139,\n",
            "                                              'step': 187},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 45, 663911, tzinfo=<UTC>),\n",
            "                                              'epoch': 50,\n",
            "                                              'mean_loss': 0.0003010271,\n",
            "                                              'step': 188},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 46, 907392, tzinfo=<UTC>),\n",
            "                                              'epoch': 50,\n",
            "                                              'mean_loss': 0.000984581,\n",
            "                                              'step': 189},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 48, 197363, tzinfo=<UTC>),\n",
            "                                              'epoch': 50,\n",
            "                                              'mean_loss': 0.0001981319,\n",
            "                                              'step': 190},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 49, 428991, tzinfo=<UTC>),\n",
            "                                              'epoch': 50,\n",
            "                                              'mean_loss': 0.00017777813,\n",
            "                                              'step': 191},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 50, 664645, tzinfo=<UTC>),\n",
            "                                              'epoch': 51,\n",
            "                                              'mean_loss': 0.00032808085,\n",
            "                                              'step': 192},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 51, 868913, tzinfo=<UTC>),\n",
            "                                              'epoch': 51,\n",
            "                                              'mean_loss': 0.00059463154,\n",
            "                                              'step': 193},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 53, 130216, tzinfo=<UTC>),\n",
            "                                              'epoch': 51,\n",
            "                                              'mean_loss': 0.0016690929,\n",
            "                                              'step': 194},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 54, 333529, tzinfo=<UTC>),\n",
            "                                              'epoch': 52,\n",
            "                                              'mean_loss': 0.00044837967,\n",
            "                                              'step': 195},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 55, 572466, tzinfo=<UTC>),\n",
            "                                              'epoch': 52,\n",
            "                                              'mean_loss': 0.0005696884,\n",
            "                                              'step': 196},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 56, 837266, tzinfo=<UTC>),\n",
            "                                              'epoch': 52,\n",
            "                                              'mean_loss': 0.0020454866,\n",
            "                                              'step': 197},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 58, 84490, tzinfo=<UTC>),\n",
            "                                              'epoch': 52,\n",
            "                                              'mean_loss': 0.008037148,\n",
            "                                              'step': 198},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 7, 59, 322229, tzinfo=<UTC>),\n",
            "                                              'epoch': 53,\n",
            "                                              'mean_loss': 0.00042736152,\n",
            "                                              'step': 199},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 0, 564415, tzinfo=<UTC>),\n",
            "                                              'epoch': 53,\n",
            "                                              'mean_loss': 0.0023432998,\n",
            "                                              'step': 200},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 1, 788247, tzinfo=<UTC>),\n",
            "                                              'epoch': 53,\n",
            "                                              'mean_loss': 0.0002715867,\n",
            "                                              'step': 201},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 3, 24660, tzinfo=<UTC>),\n",
            "                                              'epoch': 53,\n",
            "                                              'mean_loss': 0.00084764615,\n",
            "                                              'step': 202},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 4, 292818, tzinfo=<UTC>),\n",
            "                                              'epoch': 54,\n",
            "                                              'mean_loss': 0.00030835773,\n",
            "                                              'step': 203},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 5, 506320, tzinfo=<UTC>),\n",
            "                                              'epoch': 54,\n",
            "                                              'mean_loss': 0.00062151754,\n",
            "                                              'step': 204},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 6, 782668, tzinfo=<UTC>),\n",
            "                                              'epoch': 54,\n",
            "                                              'mean_loss': 3.3617685e-05,\n",
            "                                              'step': 205},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 8, 36202, tzinfo=<UTC>),\n",
            "                                              'epoch': 54,\n",
            "                                              'mean_loss': 0.00043850893,\n",
            "                                              'step': 206},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 9, 280914, tzinfo=<UTC>),\n",
            "                                              'epoch': 55,\n",
            "                                              'mean_loss': 0.0014309261,\n",
            "                                              'step': 207},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 10, 507550, tzinfo=<UTC>),\n",
            "                                              'epoch': 55,\n",
            "                                              'mean_loss': 0.00029372238,\n",
            "                                              'step': 208},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 11, 790404, tzinfo=<UTC>),\n",
            "                                              'epoch': 55,\n",
            "                                              'mean_loss': 0.000352273,\n",
            "                                              'step': 209},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 13, 28160, tzinfo=<UTC>),\n",
            "                                              'epoch': 56,\n",
            "                                              'mean_loss': 0.0034054378,\n",
            "                                              'step': 210},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 14, 285902, tzinfo=<UTC>),\n",
            "                                              'epoch': 56,\n",
            "                                              'mean_loss': 0.00038256973,\n",
            "                                              'step': 211},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 15, 554993, tzinfo=<UTC>),\n",
            "                                              'epoch': 56,\n",
            "                                              'mean_loss': 0.0029042722,\n",
            "                                              'step': 212},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 16, 782794, tzinfo=<UTC>),\n",
            "                                              'epoch': 56,\n",
            "                                              'mean_loss': 0.0013111946,\n",
            "                                              'step': 213},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 18, 18302, tzinfo=<UTC>),\n",
            "                                              'epoch': 57,\n",
            "                                              'mean_loss': 0.0005805618,\n",
            "                                              'step': 214},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 19, 256860, tzinfo=<UTC>),\n",
            "                                              'epoch': 57,\n",
            "                                              'mean_loss': 0.001555414,\n",
            "                                              'step': 215},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 20, 531737, tzinfo=<UTC>),\n",
            "                                              'epoch': 57,\n",
            "                                              'mean_loss': 0.0035115764,\n",
            "                                              'step': 216},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 21, 760323, tzinfo=<UTC>),\n",
            "                                              'epoch': 57,\n",
            "                                              'mean_loss': 0.00041363586,\n",
            "                                              'step': 217},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 23, 38254, tzinfo=<UTC>),\n",
            "                                              'epoch': 58,\n",
            "                                              'mean_loss': 8.1257895e-08,\n",
            "                                              'step': 218},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 24, 459123, tzinfo=<UTC>),\n",
            "                                              'epoch': 58,\n",
            "                                              'mean_loss': 0.0016267635,\n",
            "                                              'step': 219},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 25, 876654, tzinfo=<UTC>),\n",
            "                                              'epoch': 58,\n",
            "                                              'mean_loss': 0.0012277836,\n",
            "                                              'step': 220},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 27, 142763, tzinfo=<UTC>),\n",
            "                                              'epoch': 58,\n",
            "                                              'mean_loss': 0.0013576464,\n",
            "                                              'step': 221},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 28, 423880, tzinfo=<UTC>),\n",
            "                                              'epoch': 59,\n",
            "                                              'mean_loss': 0.0014309958,\n",
            "                                              'step': 222},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 29, 651501, tzinfo=<UTC>),\n",
            "                                              'epoch': 59,\n",
            "                                              'mean_loss': 0.00035945722,\n",
            "                                              'step': 223},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 30, 933277, tzinfo=<UTC>),\n",
            "                                              'epoch': 59,\n",
            "                                              'mean_loss': 0.0002977748,\n",
            "                                              'step': 224},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 32, 192499, tzinfo=<UTC>),\n",
            "                                              'epoch': 60,\n",
            "                                              'mean_loss': 0.0016817113,\n",
            "                                              'step': 225},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 33, 457414, tzinfo=<UTC>),\n",
            "                                              'epoch': 60,\n",
            "                                              'mean_loss': 0.0006552929,\n",
            "                                              'step': 226},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 34, 826380, tzinfo=<UTC>),\n",
            "                                              'epoch': 60,\n",
            "                                              'mean_loss': 0.00012618338,\n",
            "                                              'step': 227},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 36, 79976, tzinfo=<UTC>),\n",
            "                                              'epoch': 60,\n",
            "                                              'mean_loss': 0.00076779944,\n",
            "                                              'step': 228},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 37, 468400, tzinfo=<UTC>),\n",
            "                                              'epoch': 61,\n",
            "                                              'mean_loss': 0.0010934594,\n",
            "                                              'step': 229},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 38, 702382, tzinfo=<UTC>),\n",
            "                                              'epoch': 61,\n",
            "                                              'mean_loss': 0.0011660276,\n",
            "                                              'step': 230},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 39, 959625, tzinfo=<UTC>),\n",
            "                                              'epoch': 61,\n",
            "                                              'mean_loss': 8.885516e-06,\n",
            "                                              'step': 231},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 41, 195827, tzinfo=<UTC>),\n",
            "                                              'epoch': 61,\n",
            "                                              'mean_loss': 0.00091533817,\n",
            "                                              'step': 232},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 42, 428366, tzinfo=<UTC>),\n",
            "                                              'epoch': 62,\n",
            "                                              'mean_loss': 0.00090357044,\n",
            "                                              'step': 233},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 44, 734547, tzinfo=<UTC>),\n",
            "                                              'epoch': 62,\n",
            "                                              'mean_loss': 0.00091215235,\n",
            "                                              'step': 234},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 45, 975299, tzinfo=<UTC>),\n",
            "                                              'epoch': 62,\n",
            "                                              'mean_loss': 0.0008807101,\n",
            "                                              'step': 235},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 47, 381337, tzinfo=<UTC>),\n",
            "                                              'epoch': 62,\n",
            "                                              'mean_loss': 0.00038245833,\n",
            "                                              'step': 236},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 48, 654093, tzinfo=<UTC>),\n",
            "                                              'epoch': 63,\n",
            "                                              'mean_loss': 0.00080911745,\n",
            "                                              'step': 237},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 49, 882751, tzinfo=<UTC>),\n",
            "                                              'epoch': 63,\n",
            "                                              'mean_loss': 0.0003725673,\n",
            "                                              'step': 238},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 51, 110358, tzinfo=<UTC>),\n",
            "                                              'epoch': 63,\n",
            "                                              'mean_loss': 0.0012469762,\n",
            "                                              'step': 239},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 52, 679155, tzinfo=<UTC>),\n",
            "                                              'epoch': 64,\n",
            "                                              'mean_loss': 0.0005009441,\n",
            "                                              'step': 240},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 53, 911603, tzinfo=<UTC>),\n",
            "                                              'epoch': 64,\n",
            "                                              'mean_loss': 0.0005480327,\n",
            "                                              'step': 241},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 55, 184955, tzinfo=<UTC>),\n",
            "                                              'epoch': 64,\n",
            "                                              'mean_loss': -1.8597348e-06,\n",
            "                                              'step': 242},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 56, 578096, tzinfo=<UTC>),\n",
            "                                              'epoch': 64,\n",
            "                                              'mean_loss': 0.002287712,\n",
            "                                              'step': 243},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 57, 828452, tzinfo=<UTC>),\n",
            "                                              'epoch': 65,\n",
            "                                              'mean_loss': 6.471912e-05,\n",
            "                                              'step': 244},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 8, 59, 35379, tzinfo=<UTC>),\n",
            "                                              'epoch': 65,\n",
            "                                              'mean_loss': 0.00031904934,\n",
            "                                              'step': 245},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 0, 293344, tzinfo=<UTC>),\n",
            "                                              'epoch': 65,\n",
            "                                              'mean_loss': 1.0820688e-05,\n",
            "                                              'step': 246},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 1, 560935, tzinfo=<UTC>),\n",
            "                                              'epoch': 65,\n",
            "                                              'mean_loss': 0.0012223909,\n",
            "                                              'step': 247},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 2, 793184, tzinfo=<UTC>),\n",
            "                                              'epoch': 66,\n",
            "                                              'mean_loss': 0.0008803075,\n",
            "                                              'step': 248},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 4, 43681, tzinfo=<UTC>),\n",
            "                                              'epoch': 66,\n",
            "                                              'mean_loss': 0.0010418366,\n",
            "                                              'step': 249},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 5, 296877, tzinfo=<UTC>),\n",
            "                                              'epoch': 66,\n",
            "                                              'mean_loss': 0.0011515429,\n",
            "                                              'step': 250},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 6, 531123, tzinfo=<UTC>),\n",
            "                                              'epoch': 66,\n",
            "                                              'mean_loss': 9.024725e-05,\n",
            "                                              'step': 251},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 7, 793511, tzinfo=<UTC>),\n",
            "                                              'epoch': 67,\n",
            "                                              'mean_loss': 1.4540274e-06,\n",
            "                                              'step': 252},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 9, 25627, tzinfo=<UTC>),\n",
            "                                              'epoch': 67,\n",
            "                                              'mean_loss': 0.00012591947,\n",
            "                                              'step': 253},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 10, 275288, tzinfo=<UTC>),\n",
            "                                              'epoch': 67,\n",
            "                                              'mean_loss': 0.0008015272,\n",
            "                                              'step': 254},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 11, 522475, tzinfo=<UTC>),\n",
            "                                              'epoch': 68,\n",
            "                                              'mean_loss': 0.0008423489,\n",
            "                                              'step': 255},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 12, 764784, tzinfo=<UTC>),\n",
            "                                              'epoch': 68,\n",
            "                                              'mean_loss': 0.0011362592,\n",
            "                                              'step': 256},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 14, 58405, tzinfo=<UTC>),\n",
            "                                              'epoch': 68,\n",
            "                                              'mean_loss': 0.001567196,\n",
            "                                              'step': 257},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 15, 317351, tzinfo=<UTC>),\n",
            "                                              'epoch': 68,\n",
            "                                              'mean_loss': 0.0004869902,\n",
            "                                              'step': 258},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 16, 557683, tzinfo=<UTC>),\n",
            "                                              'epoch': 69,\n",
            "                                              'mean_loss': 0.00044071442,\n",
            "                                              'step': 259},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 17, 885247, tzinfo=<UTC>),\n",
            "                                              'epoch': 69,\n",
            "                                              'mean_loss': 0.0005702487,\n",
            "                                              'step': 260},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 19, 119570, tzinfo=<UTC>),\n",
            "                                              'epoch': 69,\n",
            "                                              'mean_loss': 0.0006433289,\n",
            "                                              'step': 261},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 20, 364831, tzinfo=<UTC>),\n",
            "                                              'epoch': 69,\n",
            "                                              'mean_loss': 0.00045972248,\n",
            "                                              'step': 262},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 21, 605030, tzinfo=<UTC>),\n",
            "                                              'epoch': 70,\n",
            "                                              'mean_loss': 0.0005696737,\n",
            "                                              'step': 263},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 22, 866124, tzinfo=<UTC>),\n",
            "                                              'epoch': 70,\n",
            "                                              'mean_loss': 0.0011296477,\n",
            "                                              'step': 264},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 24, 175216, tzinfo=<UTC>),\n",
            "                                              'epoch': 70,\n",
            "                                              'mean_loss': 0.00031165092,\n",
            "                                              'step': 265},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 25, 651260, tzinfo=<UTC>),\n",
            "                                              'epoch': 70,\n",
            "                                              'mean_loss': 0.0004441709,\n",
            "                                              'step': 266},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 27, 399823, tzinfo=<UTC>),\n",
            "                                              'epoch': 71,\n",
            "                                              'mean_loss': 0.0011137151,\n",
            "                                              'step': 267},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 28, 659278, tzinfo=<UTC>),\n",
            "                                              'epoch': 71,\n",
            "                                              'mean_loss': 0.0009634383,\n",
            "                                              'step': 268},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 29, 919579, tzinfo=<UTC>),\n",
            "                                              'epoch': 71,\n",
            "                                              'mean_loss': 9.269314e-05,\n",
            "                                              'step': 269},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 31, 271483, tzinfo=<UTC>),\n",
            "                                              'epoch': 72,\n",
            "                                              'mean_loss': 0.0005366412,\n",
            "                                              'step': 270},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 32, 557850, tzinfo=<UTC>),\n",
            "                                              'epoch': 72,\n",
            "                                              'mean_loss': 7.907781e-05,\n",
            "                                              'step': 271},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 33, 808536, tzinfo=<UTC>),\n",
            "                                              'epoch': 72,\n",
            "                                              'mean_loss': 0.00013625692,\n",
            "                                              'step': 272},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 35, 93333, tzinfo=<UTC>),\n",
            "                                              'epoch': 72,\n",
            "                                              'mean_loss': 0.00040144066,\n",
            "                                              'step': 273},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 36, 349643, tzinfo=<UTC>),\n",
            "                                              'epoch': 73,\n",
            "                                              'mean_loss': 0.000531957,\n",
            "                                              'step': 274},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 37, 601945, tzinfo=<UTC>),\n",
            "                                              'epoch': 73,\n",
            "                                              'mean_loss': 0.00064232247,\n",
            "                                              'step': 275},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 38, 855989, tzinfo=<UTC>),\n",
            "                                              'epoch': 73,\n",
            "                                              'mean_loss': 5.250331e-08,\n",
            "                                              'step': 276},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 40, 101868, tzinfo=<UTC>),\n",
            "                                              'epoch': 73,\n",
            "                                              'mean_loss': 0.00054669485,\n",
            "                                              'step': 277},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 41, 311587, tzinfo=<UTC>),\n",
            "                                              'epoch': 74,\n",
            "                                              'mean_loss': 0.0001149911,\n",
            "                                              'step': 278},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 42, 504293, tzinfo=<UTC>),\n",
            "                                              'epoch': 74,\n",
            "                                              'mean_loss': 0.0006582851,\n",
            "                                              'step': 279},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 44, 606424, tzinfo=<UTC>),\n",
            "                                              'epoch': 74,\n",
            "                                              'mean_loss': 0.00040794525,\n",
            "                                              'step': 280},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 45, 885918, tzinfo=<UTC>),\n",
            "                                              'epoch': 74,\n",
            "                                              'mean_loss': 0.00040962768,\n",
            "                                              'step': 281},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 47, 88903, tzinfo=<UTC>),\n",
            "                                              'epoch': 75,\n",
            "                                              'mean_loss': 0.00083329267,\n",
            "                                              'step': 282},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 48, 353963, tzinfo=<UTC>),\n",
            "                                              'epoch': 75,\n",
            "                                              'mean_loss': 0.0011541949,\n",
            "                                              'step': 283},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 49, 634966, tzinfo=<UTC>),\n",
            "                                              'epoch': 75,\n",
            "                                              'mean_loss': 0.00031403676,\n",
            "                                              'step': 284},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 50, 872769, tzinfo=<UTC>),\n",
            "                                              'epoch': 76,\n",
            "                                              'mean_loss': 0.0001571991,\n",
            "                                              'step': 285},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 52, 132997, tzinfo=<UTC>),\n",
            "                                              'epoch': 76,\n",
            "                                              'mean_loss': 3.2733406e-05,\n",
            "                                              'step': 286},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 53, 373001, tzinfo=<UTC>),\n",
            "                                              'epoch': 76,\n",
            "                                              'mean_loss': 9.974141e-06,\n",
            "                                              'step': 287},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 54, 731122, tzinfo=<UTC>),\n",
            "                                              'epoch': 76,\n",
            "                                              'mean_loss': 0.00092323206,\n",
            "                                              'step': 288},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 56, 5106, tzinfo=<UTC>),\n",
            "                                              'epoch': 77,\n",
            "                                              'mean_loss': 0.00027380337,\n",
            "                                              'step': 289},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 57, 958533, tzinfo=<UTC>),\n",
            "                                              'epoch': 77,\n",
            "                                              'mean_loss': 0.000730188,\n",
            "                                              'step': 290},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 9, 59, 218392, tzinfo=<UTC>),\n",
            "                                              'epoch': 77,\n",
            "                                              'mean_loss': 8.784898e-05,\n",
            "                                              'step': 291},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 0, 458718, tzinfo=<UTC>),\n",
            "                                              'epoch': 77,\n",
            "                                              'mean_loss': 0.00035266066,\n",
            "                                              'step': 292},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 1, 678532, tzinfo=<UTC>),\n",
            "                                              'epoch': 78,\n",
            "                                              'mean_loss': 0.00035272574,\n",
            "                                              'step': 293},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 2, 925032, tzinfo=<UTC>),\n",
            "                                              'epoch': 78,\n",
            "                                              'mean_loss': 0.00036251757,\n",
            "                                              'step': 294},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 4, 185515, tzinfo=<UTC>),\n",
            "                                              'epoch': 78,\n",
            "                                              'mean_loss': 0.00063127023,\n",
            "                                              'step': 295},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 5, 450029, tzinfo=<UTC>),\n",
            "                                              'epoch': 78,\n",
            "                                              'mean_loss': 0.00056570326,\n",
            "                                              'step': 296},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 6, 669796, tzinfo=<UTC>),\n",
            "                                              'epoch': 79,\n",
            "                                              'mean_loss': 0.00019644247,\n",
            "                                              'step': 297},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 7, 912769, tzinfo=<UTC>),\n",
            "                                              'epoch': 79,\n",
            "                                              'mean_loss': 1.2713252e-05,\n",
            "                                              'step': 298},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 9, 283472, tzinfo=<UTC>),\n",
            "                                              'epoch': 79,\n",
            "                                              'mean_loss': 0.00024443713,\n",
            "                                              'step': 299},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 10, 628408, tzinfo=<UTC>),\n",
            "                                              'epoch': 80,\n",
            "                                              'mean_loss': 0.001333565,\n",
            "                                              'step': 300},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 11, 890583, tzinfo=<UTC>),\n",
            "                                              'epoch': 80,\n",
            "                                              'mean_loss': 0.0007013555,\n",
            "                                              'step': 301},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 13, 592038, tzinfo=<UTC>),\n",
            "                                              'epoch': 80,\n",
            "                                              'mean_loss': 2.6616384e-05,\n",
            "                                              'step': 302},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 14, 822838, tzinfo=<UTC>),\n",
            "                                              'epoch': 80,\n",
            "                                              'mean_loss': 0.0005623731,\n",
            "                                              'step': 303},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 16, 205657, tzinfo=<UTC>),\n",
            "                                              'epoch': 81,\n",
            "                                              'mean_loss': 0.00032505486,\n",
            "                                              'step': 304},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 18, 71680, tzinfo=<UTC>),\n",
            "                                              'epoch': 81,\n",
            "                                              'mean_loss': 0.0005101152,\n",
            "                                              'step': 305},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 19, 294720, tzinfo=<UTC>),\n",
            "                                              'epoch': 81,\n",
            "                                              'mean_loss': 0.0010035196,\n",
            "                                              'step': 306},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 20, 524938, tzinfo=<UTC>),\n",
            "                                              'epoch': 81,\n",
            "                                              'mean_loss': 0.00033056142,\n",
            "                                              'step': 307},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 21, 762389, tzinfo=<UTC>),\n",
            "                                              'epoch': 82,\n",
            "                                              'mean_loss': 6.966456e-05,\n",
            "                                              'step': 308},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 23, 25594, tzinfo=<UTC>),\n",
            "                                              'epoch': 82,\n",
            "                                              'mean_loss': 0.00037358585,\n",
            "                                              'step': 309},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 24, 257330, tzinfo=<UTC>),\n",
            "                                              'epoch': 82,\n",
            "                                              'mean_loss': 0.0002650607,\n",
            "                                              'step': 310},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 25, 497375, tzinfo=<UTC>),\n",
            "                                              'epoch': 82,\n",
            "                                              'mean_loss': 0.0009153653,\n",
            "                                              'step': 311},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 26, 713095, tzinfo=<UTC>),\n",
            "                                              'epoch': 83,\n",
            "                                              'mean_loss': 0.0009658756,\n",
            "                                              'step': 312},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 27, 973220, tzinfo=<UTC>),\n",
            "                                              'epoch': 83,\n",
            "                                              'mean_loss': 0.00014872942,\n",
            "                                              'step': 313},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 29, 226304, tzinfo=<UTC>),\n",
            "                                              'epoch': 83,\n",
            "                                              'mean_loss': 9.968644e-06,\n",
            "                                              'step': 314},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 30, 493632, tzinfo=<UTC>),\n",
            "                                              'epoch': 84,\n",
            "                                              'mean_loss': 0.00027738986,\n",
            "                                              'step': 315},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 31, 735723, tzinfo=<UTC>),\n",
            "                                              'epoch': 84,\n",
            "                                              'mean_loss': 0.0004675896,\n",
            "                                              'step': 316},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 32, 954139, tzinfo=<UTC>),\n",
            "                                              'epoch': 84,\n",
            "                                              'mean_loss': 0.00014443416,\n",
            "                                              'step': 317},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 34, 196120, tzinfo=<UTC>),\n",
            "                                              'epoch': 84,\n",
            "                                              'mean_loss': 0.0006946635,\n",
            "                                              'step': 318},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 35, 443514, tzinfo=<UTC>),\n",
            "                                              'epoch': 85,\n",
            "                                              'mean_loss': 0.0007360133,\n",
            "                                              'step': 319},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 36, 687667, tzinfo=<UTC>),\n",
            "                                              'epoch': 85,\n",
            "                                              'mean_loss': 1.326669e-06,\n",
            "                                              'step': 320},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 37, 903099, tzinfo=<UTC>),\n",
            "                                              'epoch': 85,\n",
            "                                              'mean_loss': 0.0005314335,\n",
            "                                              'step': 321},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 39, 138629, tzinfo=<UTC>),\n",
            "                                              'epoch': 85,\n",
            "                                              'mean_loss': 6.947189e-05,\n",
            "                                              'step': 322},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 40, 371666, tzinfo=<UTC>),\n",
            "                                              'epoch': 86,\n",
            "                                              'mean_loss': 0.00053617253,\n",
            "                                              'step': 323},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 41, 602304, tzinfo=<UTC>),\n",
            "                                              'epoch': 86,\n",
            "                                              'mean_loss': 9.1956696e-05,\n",
            "                                              'step': 324},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 42, 836714, tzinfo=<UTC>),\n",
            "                                              'epoch': 86,\n",
            "                                              'mean_loss': 0.00018627953,\n",
            "                                              'step': 325},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 44, 874288, tzinfo=<UTC>),\n",
            "                                              'epoch': 86,\n",
            "                                              'mean_loss': 0.0002088271,\n",
            "                                              'step': 326},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 46, 162027, tzinfo=<UTC>),\n",
            "                                              'epoch': 87,\n",
            "                                              'mean_loss': 0.00075449655,\n",
            "                                              'step': 327},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 47, 431864, tzinfo=<UTC>),\n",
            "                                              'epoch': 87,\n",
            "                                              'mean_loss': 7.8588026e-05,\n",
            "                                              'step': 328},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 48, 680277, tzinfo=<UTC>),\n",
            "                                              'epoch': 87,\n",
            "                                              'mean_loss': -1.3336539e-06,\n",
            "                                              'step': 329},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 49, 968707, tzinfo=<UTC>),\n",
            "                                              'epoch': 88,\n",
            "                                              'mean_loss': 0.00012271712,\n",
            "                                              'step': 330},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 51, 259023, tzinfo=<UTC>),\n",
            "                                              'epoch': 88,\n",
            "                                              'mean_loss': 0.0017514592,\n",
            "                                              'step': 331},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 52, 511807, tzinfo=<UTC>),\n",
            "                                              'epoch': 88,\n",
            "                                              'mean_loss': 4.1678373e-05,\n",
            "                                              'step': 332},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 53, 808914, tzinfo=<UTC>),\n",
            "                                              'epoch': 88,\n",
            "                                              'mean_loss': 0.0006764167,\n",
            "                                              'step': 333},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 55, 179417, tzinfo=<UTC>),\n",
            "                                              'epoch': 89,\n",
            "                                              'mean_loss': 0.00013730745,\n",
            "                                              'step': 334},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 56, 395678, tzinfo=<UTC>),\n",
            "                                              'epoch': 89,\n",
            "                                              'mean_loss': 0.00032095844,\n",
            "                                              'step': 335},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 57, 654428, tzinfo=<UTC>),\n",
            "                                              'epoch': 89,\n",
            "                                              'mean_loss': 0.00015303271,\n",
            "                                              'step': 336},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 10, 58, 891717, tzinfo=<UTC>),\n",
            "                                              'epoch': 89,\n",
            "                                              'mean_loss': 0.00012956047,\n",
            "                                              'step': 337},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 0, 151925, tzinfo=<UTC>),\n",
            "                                              'epoch': 90,\n",
            "                                              'mean_loss': 7.675003e-05,\n",
            "                                              'step': 338},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 1, 398761, tzinfo=<UTC>),\n",
            "                                              'epoch': 90,\n",
            "                                              'mean_loss': 0.00044489285,\n",
            "                                              'step': 339},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 2, 641274, tzinfo=<UTC>),\n",
            "                                              'epoch': 90,\n",
            "                                              'mean_loss': 1.4036312e-05,\n",
            "                                              'step': 340},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 3, 883546, tzinfo=<UTC>),\n",
            "                                              'epoch': 90,\n",
            "                                              'mean_loss': 0.00015219976,\n",
            "                                              'step': 341},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 5, 141535, tzinfo=<UTC>),\n",
            "                                              'epoch': 91,\n",
            "                                              'mean_loss': 0.00018677826,\n",
            "                                              'step': 342},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 6, 373784, tzinfo=<UTC>),\n",
            "                                              'epoch': 91,\n",
            "                                              'mean_loss': 0.000115128816,\n",
            "                                              'step': 343},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 7, 645535, tzinfo=<UTC>),\n",
            "                                              'epoch': 91,\n",
            "                                              'mean_loss': 0.00039966288,\n",
            "                                              'step': 344},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 8, 884670, tzinfo=<UTC>),\n",
            "                                              'epoch': 92,\n",
            "                                              'mean_loss': 0.0001351597,\n",
            "                                              'step': 345},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 10, 149429, tzinfo=<UTC>),\n",
            "                                              'epoch': 92,\n",
            "                                              'mean_loss': 6.1459374e-05,\n",
            "                                              'step': 346},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 11, 448963, tzinfo=<UTC>),\n",
            "                                              'epoch': 92,\n",
            "                                              'mean_loss': 0.00023051281,\n",
            "                                              'step': 347},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 12, 693703, tzinfo=<UTC>),\n",
            "                                              'epoch': 92,\n",
            "                                              'mean_loss': 0.00078510307,\n",
            "                                              'step': 348},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 13, 938911, tzinfo=<UTC>),\n",
            "                                              'epoch': 93,\n",
            "                                              'mean_loss': 8.103554e-06,\n",
            "                                              'step': 349},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 15, 460723, tzinfo=<UTC>),\n",
            "                                              'epoch': 93,\n",
            "                                              'mean_loss': 0.0019005266,\n",
            "                                              'step': 350},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 16, 766511, tzinfo=<UTC>),\n",
            "                                              'epoch': 93,\n",
            "                                              'mean_loss': 6.863149e-06,\n",
            "                                              'step': 351},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 18, 27508, tzinfo=<UTC>),\n",
            "                                              'epoch': 93,\n",
            "                                              'mean_loss': 0.0002926389,\n",
            "                                              'step': 352},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 19, 286497, tzinfo=<UTC>),\n",
            "                                              'epoch': 94,\n",
            "                                              'mean_loss': 0.00013998023,\n",
            "                                              'step': 353},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 20, 556785, tzinfo=<UTC>),\n",
            "                                              'epoch': 94,\n",
            "                                              'mean_loss': 2.2997847e-05,\n",
            "                                              'step': 354},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 21, 804735, tzinfo=<UTC>),\n",
            "                                              'epoch': 94,\n",
            "                                              'mean_loss': 0.0005936278,\n",
            "                                              'step': 355},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 23, 22367, tzinfo=<UTC>),\n",
            "                                              'epoch': 94,\n",
            "                                              'mean_loss': 3.43258e-05,\n",
            "                                              'step': 356},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 24, 284890, tzinfo=<UTC>),\n",
            "                                              'epoch': 95,\n",
            "                                              'mean_loss': 0.00010312116,\n",
            "                                              'step': 357},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 25, 525734, tzinfo=<UTC>),\n",
            "                                              'epoch': 95,\n",
            "                                              'mean_loss': 0.00015714776,\n",
            "                                              'step': 358},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 26, 780375, tzinfo=<UTC>),\n",
            "                                              'epoch': 95,\n",
            "                                              'mean_loss': 5.73016e-05,\n",
            "                                              'step': 359},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 28, 23466, tzinfo=<UTC>),\n",
            "                                              'epoch': 96,\n",
            "                                              'mean_loss': 0.00012817327,\n",
            "                                              'step': 360},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 29, 268204, tzinfo=<UTC>),\n",
            "                                              'epoch': 96,\n",
            "                                              'mean_loss': 3.9030332e-05,\n",
            "                                              'step': 361},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 30, 507390, tzinfo=<UTC>),\n",
            "                                              'epoch': 96,\n",
            "                                              'mean_loss': 0.0005360425,\n",
            "                                              'step': 362},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 31, 727121, tzinfo=<UTC>),\n",
            "                                              'epoch': 96,\n",
            "                                              'mean_loss': 0.00017444952,\n",
            "                                              'step': 363},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 33, 34563, tzinfo=<UTC>),\n",
            "                                              'epoch': 97,\n",
            "                                              'mean_loss': 0.0010171408,\n",
            "                                              'step': 364},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 34, 447584, tzinfo=<UTC>),\n",
            "                                              'epoch': 97,\n",
            "                                              'mean_loss': 0.0004899306,\n",
            "                                              'step': 365},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 35, 699821, tzinfo=<UTC>),\n",
            "                                              'epoch': 97,\n",
            "                                              'mean_loss': 0.00017226115,\n",
            "                                              'step': 366},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 36, 936549, tzinfo=<UTC>),\n",
            "                                              'epoch': 97,\n",
            "                                              'mean_loss': 4.2724423e-07,\n",
            "                                              'step': 367},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 38, 203783, tzinfo=<UTC>),\n",
            "                                              'epoch': 98,\n",
            "                                              'mean_loss': 1.9560219e-05,\n",
            "                                              'step': 368},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 39, 464518, tzinfo=<UTC>),\n",
            "                                              'epoch': 98,\n",
            "                                              'mean_loss': 0.00011098804,\n",
            "                                              'step': 369},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 40, 721560, tzinfo=<UTC>),\n",
            "                                              'epoch': 98,\n",
            "                                              'mean_loss': 0.0005288075,\n",
            "                                              'step': 370},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 41, 968386, tzinfo=<UTC>),\n",
            "                                              'epoch': 98,\n",
            "                                              'mean_loss': 4.2606727e-05,\n",
            "                                              'step': 371},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 43, 207703, tzinfo=<UTC>),\n",
            "                                              'epoch': 99,\n",
            "                                              'mean_loss': 1.1964934e-05,\n",
            "                                              'step': 372},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 45, 297458, tzinfo=<UTC>),\n",
            "                                              'epoch': 99,\n",
            "                                              'mean_loss': 0.00035788305,\n",
            "                                              'step': 373},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 46, 714760, tzinfo=<UTC>),\n",
            "                                              'epoch': 99,\n",
            "                                              'mean_loss': 7.525133e-05,\n",
            "                                              'step': 374},\n",
            "                                             {'compute_time': datetime.datetime(2023, 9, 19, 19, 11, 48, 17671, tzinfo=<UTC>),\n",
            "                                              'epoch': 100,\n",
            "                                              'mean_loss': 5.5355486e-06,\n",
            "                                              'step': 375}],\n",
            "                                  hyperparameters=Hyperparameters(epoch_count=100,\n",
            "                                                                  batch_size=4,\n",
            "                                                                  learning_rate=0.001)))\n"
          ]
        }
      ],
      "source": [
        "model = genai.get_tuned_model(f'tunedModels/{name}')\n",
        "\n",
        "model"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "G9M2pb1_xZQH"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'This is my model.'"
            ]
          },
          "execution_count": 23,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "model.description"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "i_TpwvBB4bQ7"
      },
      "source": [
        "## Delete the model\n",
        "\n",
        "You can clean up your tuned model list by deleting models you no longer need. Use the `genai.delete_tuned_model` method to delete a model. If you canceled any tuning jobs, you may want to delete those as their performance may be unpredictable."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "kN_bkut_4ayL"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "<class 'google.api_core.exceptions.NotFound'>: 404 Tuned model tunedModels/generate-num-4668 does not exist.\n"
          ]
        }
      ],
      "source": [
        "genai.delete_tuned_model(f'tunedModels/{name}')\n",
        "\n",
        "try:\n",
        "  m = genai.get_tuned_model(f'tunedModels/{name}')\n",
        "  print(m)\n",
        "except Exception as e:\n",
        "  print(f\"{type(e)}: {e}\")"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "name": "tuning_quickstart_python.ipynb",
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}
